Bash Tips

Bash multiprocessing

Supercharge Your Bash Scripts with Multiprocessing
Fr1nge's Personal Blog

Useful commands

cheat.sh

​cheat.sh provides unified access to the best community driven cheat sheets repositories of the world.
1
# To get a cheat sheet just use the following command
2
# Change <command> to command name, for instance, ls, id, curl, etc.
3
$ curl cheat.sh/<command>
4
​
5
# Cheat sheet for curl
6
$ curl cheat.sh/curl
Copied!

aria2c

​aria2 is a lightweight multi-protocol & multi-source, cross platform download utility operated in command-line.
1
# Multi-thread downloading
2
$ aria2c -x5 <URL>
3
​
4
# Restarting aria2c continues unfinished download
5
$ aria2c -x5 <URL>
6
^C
7
$ aria2c -x5 <URL> # downloading continues
8
​
9
# Download torrent files (just pass the torrent file to the input)
10
$ aria2c <name>.torrent
Copied!

chmod

1
# Enable all permissions for the owner, write permissions to the group and execute permissions to others
2
# or rwx-w---x
3
$ chmod 721 <name>
4
​
5
### Verbose form
6
# Enable user for rwx
7
$ chmod u+rwx <name>
8
​
9
# Enable group for w
10
$ chmod g+w <name>
11
​
12
# Enable others for x
13
$ chmod o+x <name>
14
​
15
# Enable everyone for x
16
$ chmod a+x <file>
17
​
18
### Remove permissions (use '-' instead of '+')
19
# Disable group and others for x
20
$ chmod og-x <name>
21
​
22
### setuid (4), setgid (2), sticky (1) bits
23
# Set setgid bit
24
# or rwxr-sr-x
25
$ chmod 2755 <name>
26
​
27
## Verbose form
28
# Set setuid bit
29
$ chmod u+s <name>
30
​
31
# Set setgid bit
32
$ chmod g+s <name>
33
​
34
# Set sticky bit
35
$ chmod o+t <name>
Copied!

curl

​curl a command line tool and library for transferring data with URL syntax.
Useful operation mode:
  • ​-o, --output - write output to file instead of stdout.
  • ​-X, --request - specifies a custom request method to use when communicating with the HTTP server.
  • ​-H, --header ​header/@file - extra header to include in the request when sending HTTP to a server.
  • ​-d, --data - adds body to the request.
  • ​--path-as-is - tell curl to not handle sequences of /../ or /./ in the given URL path.
  • ​-v, --verbose - makes curl verbose during the operation.

find

1
# Case sensitive search
2
$ find / -name '*some*'
3
​
4
# Case insensitive search
5
$ find / -iname '*some*'
6
​
7
# ls style output formatting
8
$ find / -iname '*some*' -ls
9
​
10
# Delete found files (danger: there is no confirmation)
11
$ find dir_to_delete/ -delete
12
​
13
# Executing a script with search results
14
# Format:
15
# -exec <command> {} \;
16
# <commad> - script/command to execute
17
# {} - place for the found file
18
# \; - end of <command>
19
$ find / -iname '*some*' -exec ./script.sh {} \;
Copied!

grep

1
# Search string without using regex
2
# fgrep is an alias for grep -F
3
$ fgrep
4
​
5
# Using perl-compatible regex (or "real" regex)
6
$ grep -P
7
​
8
# Searching inside gz archives
9
$ zgrep
10
​
11
# Highlight found words in search results
12
$ grep --color=force
13
​
14
# Invert the sense of matching
15
$ grep -v
16
​
17
# grep by file contents
18
$ grep -rnw '/path/to/somewhere/' -e 'pattern'
Copied!

ipython

​ipython is a handy command shell for python, which supports:
  • Tab completion
  • Help by .method? + enter
  • Embedding in a script for debugging using the interactive shell:
    1
    from IPython import embed; embed()
    Copied!

ncdu

​ncdu is a disk usage analyzer with an ncurses interface.
1
$ ncdu /
Copied!
Useful hotkeys:
  • Navigation by arrows
  • s - sort by size
  • C - sort by quantity
  • c - show quantity
  • d - delete

pv

​pv - monitor the progress of data through a pipe.
1
$ ./app1 | pv | ./app2
2
$ cat /dev/urandom | pv | xxd > /dev/null
3
​
4
# cat-like behavior with a progress bar
5
$ pv file | ./app
6
$ pv some_file.txt | bzip2 > /dev/null
7
​
8
# tar compression progress bar
9
$ tar cz /folder | pv > folder.tar.gz
10
​
11
# Network data transfer progress bar
12
$ pv folder.tar.gz | nc -nlvp 1337
13
​
14
# Monitor other process
15
$ pv -d PID
16
​
17
# Limit speed
18
$ pv file.txt -L 2 # 2 bytes per second
19
$ pv file.txt -L -l 2 # 2 lines per second
Copied!

tar

​tar saves many files together into a single tape or disk archive, and can restore individual files from the archive.
Useful operation mode:
  • -c - create a new archive
  • -f - use archive file or device ARCHIVE
  • -j, --bzip2 - compress/decompress the archive through bzip2
  • -z, --gzip - compress/decompress the archive through gzip
  • -t, --list - list the contents of an archive
  • -x - extract files from an archive
  • -C, --directory=DIR - change output directory to DIR
  • -v verbosely list files processed
1
# Create tar archive
2
$ tar cf archive.tar foo bar
3
​
4
# Create bzip2 archive
5
$ tar cf archive.tar.bz2 foo bar
6
​
7
# Create gzip archive
8
$ tar cf archive.tar.gz foo bar
9
​
10
# Extract archive
11
$ tar xf archive.tar.gz
Copied!

youtube-dl

​youtube-dl is a command-line program to download videos from youtube and other video sites.
1
$ youtube-dl https://youtu.be/somth12asd34
Copied!

Modern Unix

GitHub - ibraheemdev/modern-unix: A collection of modern/faster/saner alternatives to common unix commands.
GitHub