Bash

Curl Command line commands

curl --trace-ascii debugdump.txt http://www.example.com/ # save output as file curl -o filename.html http://www.example.com/ # follow redirect curl -L yahoo.com # Put request with header curl -b cookie.txt \ -X PUT \ # @test.png is a variable --data-binary "@test.png" \ -H "Content-Type: image/png" \ $URL \ -O

IntelliJ IDE

Settings Shortcuts Shortcut Function Shift + F1 Lookup external documentation CTRL + ALT + Shift + S Project settings Setup external documentation In the Project Structure dialogCtrl+Shift+Alt+S, select SDKs. Select the necessary JDK version if you have several JDKs configured, and open the Documentation Path tab on the right. Click the Specify URL icon and enter the external documentation URL.

Remote server connections

# Make connection with server sftp -o User=me@mail.com myserver.com:/path/to/files get/put # copy folder to remote location scp -r . user@host:/path

Cron related Info

Crontab Script to run #!/bin/bash #script1.sh tmpFile=$(mktemp) date=$(date +"%Y-%m-%d") /cm/shared/apps/bioinf/R/3.5.1/bin/Rscript /path/to/script.R >>$tmpFile if [ $? == 0 ];then echo "Script succeeded" >> $tmpFile f1="/path/to/output.pdf" mail -s "Feedback script1 $date" -a $f1 me@email.com < $tmpFile else echo "Script failed" >> $tmpFile mail -s "Feedback script1 $date" me@email.com < $tmpFile fi # Use specific separator for awk columns # Minute Hour Day Month DayOfTheWeek # Two times every hour during office hours 10,30 8-16 * * * /home/me/script1.

Bash string functions

https://www.thegeekstuff.com/2010/07/bash-string-manipulation/ #shortest substring match ${string#substring} #shortest substring match (backwards) ${string%substring}

Linux housekeeping

#system info cat /etc/os-release uname -a Sudo User_Alias ADMINS = z691172 ADMINS ALL = NOPASSWD: /usr/bin/apt-get # firewall sudo ufw disable/enable/status

Bash Utilities

Argument checking and parsing in scripts display_usage() { echo "Not enough args specified. Plese specify 1) sampleId and 2)bamfile" echo -e "\nUsage:\n$(basename "$0") [args] {sampleId} {bamfile} [optional args] {targetfile}\n" } if [ $# -lt 2 ];then display_usage exit 1 fi Use flags to makereport -u jsmith -p notebooks -d 10-20-2011 -f pdf # colon after flag indicates obligatory args, without are optional while getopts u:d:p:f: option do case "${option}" in u) USER=${OPTARG};; d) DATE=${OPTARG};; p) PRODUCT=${OPTARG};; f) FORMAT=${OPTARG};; esac done

Rsync

R flags R flags # sync the content of source to target rsync -hvrPt /path/to/source /path/to/target #flags # -h: human readable numbers # -v: verbose # -r: recurse into directories # -P: --partial (keep partially transferred files) + # --progress (show progress during transfer) # -t: preserve modification times # --update: only newer # sync the content of source to target/source rsync /path/to/source /path/to/target/

Docker commands

Ref documentation https://docs.docker.com/v17.12/get-started/part2/#dockerfile Dockerfile #Refer to base image to start from FROM ubuntu:latest # Set environment ENV HUGO_VERSION 0.50 # Run command line command RUN apt-get update && apt-get install mytool # Download online resource to image location ADD https://github.com/spf13/hugo/releases/download/v${HUGO_VERSION}/${HUGO_BINARY}.tar.gz /usr/local/hugo/ # Open port EXPOSE 1313 # set working dir WORKDIR /app # Execute command line command CMD hugo version Docker Images #Build image from Dockerfile docker build -t my-tag dirname-with-dockerfile #Remove existing docker image docker rmi my-docker-image #Export/Import image to/from tar docker export -o my-export.

Awk Command line commands

# Use specific separator for awk columns awk -v OFS='\t' '{print $5, $1}' # Add extry entry to each processed line awk '{print $0, "extra entry"}' # Conditional printing of lines of text (e.g print lines of myFile.txt if values in 5th columm are not ".") awk '{ if ($5 != ".") { print } }' myFile.txt # calc sum of difference two columns cat Regions.bed | awk -F'\t' 'BEGIN{SUM=0}{ SUM+=$3-$2 }END{print SUM}' cat twist_swift_GI.

General Command line commands

Manipulating standard out # show all except first line of a file cat $file | tail -n+2 # show all except last row of a file cat $file | head -n -1 # show only first three rows cat $file | head -3 # show only last three rows cat $file | tail -3 # display recent changes log file tail -f $file # Remove pattern at beginning of variable "${var#http://}" # Remove pattern at end of variable "${var%/score/}" File comparison # Compare 2 files and output only overlapping rows diff -u a.

Finders

Find and execute Find and execute # find all files in subfolders of /dir1 and move to /dir find /dir1 -mindepth 2 -type f -exec mv -t /dir1 -i '{}' + #find ascii files containing db find . -name "*.R" -print | xargs grep "db" # find files newer than (m = mod time t = ref interpreted as directly as time) find . -newermt "2016-12-06" # find files created on 14 october find .

Git Command line commands

Commit management # print status of repo git status # add and stage all files git add --all # commit all changes to local repo git commit -am "$message" # Push local changes to remote repo git push # Push local changes of branch to remote repo branch git push origin $branchname:$remote_branchname # copy changes to stash git stash # Apply stash to branch git stash apply # remove last commit git reset HEAD^ # remove commit locally; on windows git reset HEAD~1 git push origin +HEAD # force-push the new HEAD commit # manipulate the last two commits interactively git rebase -i HEAD~2 # Discard local changes git checkout -- .

grep command line commands

# get pattern grep "pattern" # invert grep (i.e. only show no matches) grep -v "pattern" # search for different words grep -e 'cat|dog|bird' # Patterns # Get all characters until first dash and add dash to it /(^[^-]+)-/