Re: [PATCH 1/1] lib: remove leading spaces before tabs

[Andy Shevchenko]

On Wed, Jun 09, 2021 at 02:44:54PM -0700, Joe Perches wrote:
> On Wed, 2021-06-09 at 13:30 +0300, Andy Shevchenko wrote:
> > On Wed, Jun 9, 2021 at 9:21 AM Leizhen (ThunderTown)
> > <thunder.leizhen@xxxxxxxxxx> wrote:
> > > On 2021/6/9 13:15, Leizhen (ThunderTown) wrote:
> > > > On 2021/6/9 0:03, Joe Perches wrote:
> > > > > On Tue, 2021-06-08 at 17:00 +0800, Leizhen (ThunderTown) wrote:
> > > > > > On 2021/6/8 16:44, Andy Shevchenko wrote:
> > > > > > > On Tue, Jun 8, 2021 at 10:14 AM Zhen Lei <thunder.leizhen@xxxxxxxxxx> wrote:
> > > > > > > > 
> > > > > > > > 1) Run the following command to find and remove the leading spaces before
> > > > > > > >    tabs:
> > > > > > > >    find lib/ -type f | xargs sed -r -i 's/^[ ]+\t/\t/'
> > > > > > > 
> > > > > > > Hint for the future, try to use what Git provides, for example `git
> > > > > > > ls-files -- lib/`.
> > > > > > 
> > > > > > Okay, thanks. I learned a new trick.
> > 
> > > > > For instance:
> > > > > 
> > > > > $ git diff --stat lib
> > > > > $ git ls-files lib/ | \
> > > > >   xargs ./scripts/checkpatch.pl --types=SPACE_BEFORE_TAB --fix-inplace
> > > 
> > > I just tried it. It's too slow.
> > 
> > If checkpath accepts infinite positional arguments, then proper way of
> > use (that's how I do with simple perl regexps) is to
> > 
> > scripts/checkpatch.pl --types=SPACE_BEFORE_TAB --fix-inplace -- $(git ls-files lib/)
> 
> That won't always work:
> 
> $ git ls-files | xargs | wc -c
> 2716737
> 
> Nothing accepts infinite positional arguments.
> You are always limited by the maximum length of a command line
> 
> $ getconf ARG_MAX
> 2097152
> 
> xargs has:
> 
> $ xargs --show-limits
> Your environment variables take up 3517 bytes
> POSIX upper limit on argument length (this system): 2091587
> POSIX smallest allowable upper limit on argument length (all systems): 4096
> Maximum length of command we could actually use: 2088070
> Size of command buffer we are actually using: 131072
> Maximum parallelism (--max-procs must be no greater): 2147483647

If you care about parallelism, the checkpatch should be doing it itself.

Currently it's one perl interpreter with a single queue vs. zillions of perl
invocations (with all downsides of this). And I'm not sure which one is a
winning.

And yes, I'm aware of why xargs is ever exists (including handling of the
special file names).

> > Seems like we have a shell lesson :-)
> 
> Maybe so.
> 
> Using xargs allows use of -P to invoke parallelism.
> Or you could just use gnu parallel instead of xargs.
> 
> Using './scripts/checkpatch.pl -- $(git ls-files <path>)' will only
> allow a single process to be invoked for the files to be scanned.

-- 
With Best Regards,
Andy Shevchenko