CRUX

Welcome to CRUX bug tracking.
Tasklist

FS#1597 - pkgmk: faster strip_files() function

Attached to Project: CRUX
Opened by Fun (fun) - Saturday, 24 February 2018, 16:30 GMT
Last edited by Predrag Ivanovic (pedja) - Monday, 18 March 2019, 15:01 GMT
Task Type Improvement
Category tools → pkgutils
Status New
Assigned To CRUX Developers (crux)
Operating System CRUX
Severity Low
Priority Normal
Reported Version 3.3
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No

Details

I've attached a patch to run 'file' in parallel (using xargs -P$(nproc)).

The time spent changed from 30 seconds to .2 seconds on a 5500 files port (most of them 'data') with a faster drive. I didn't test more ports.
   0001-faster-strip_files-funct... (2.8 KiB)
This task depends upon

Comment by Fun (fun) - Saturday, 24 March 2018, 17:36 GMT
The last xargs should be run without -P$N, because the strip command (without -o) creates a temporary file with a "pseudo-random" name. These names can overlap when stripping multiple files from the same directory in parallel (e.g. mesa3d).
Comment by Fredrik Rinnestam (frinnst) - Sunday, 25 March 2018, 01:10 GMT
testing now
Comment by Fun (fun) - Saturday, 21 April 2018, 16:01 GMT
Fredrik, you really want to change the last patch line from "| xargs -r -L1 -P$N strip" to "| xargs -r -L1 strip".
https://crux.nu/gitweb/?p=tools/pkgutils.git;a=commitdiff;h=ece7f90bf8b1fb747e4607cdfb054ef5d22a13e5

Also, the previous commit https://crux.nu/gitweb/?p=tools/pkgutils.git;a=commitdiff;h=d51cdd1c8f24ef6f53cb87a0d06f4066cd91af8c removed 'bz2' from documentation (see  FS#1638 ).

Loading...