Working with the terminal

Från Wikipedia, den fria encyklopedin.

Back to main Linux page

Innehåll

[edit]

D-Bus (inter process communication

In KDE (amongst others) dbus is used to communicate between ongoing processes. Dbus can be used to send commands to a running program or to get information from it. Some examples: 

qdbus org.ktorrent.ktorrent /core org.ktorrent.core.groups

Will list all configured groups in KTorrent. 

qdbus org.ktorrent.ktorrent /core org.ktorrent.core.loadSilently file:///home/svenne/ubuntu.torrent myOsGroup

Will load ubuntu.torrent into KTorrent and add it to group myOsGroup.

More info:

Hints:

  • If running from ssh prompt (or cron-script) make sure DISPLAY variable is set (e.g. to ":0")
[edit]

CD Ripping

Here is a very convenient method to rip a bunch of audio CDs:

  1. install package abcde
  2. use mp3 config from here: http://www.andrews-corner.org/abcde.html
  3. modify config file to your liking (see man abcde). I changed the following:
    1. added INTERACTIVE=n
    2. changed OUTPUTFORMAT='${OUTPUT}/${ARTISTFILE}-${ALBUMFILE}/${TRACKNUM} ${ARTISTFILE} - ${TRACKFILE}'
  1. run using while [ true ]; do abcde; sleep 5; done

The tray will be automatically ejected when CD is ripped. All you have to do is insert a new disc and it will start automatically!

[edit]

Trim/cut mp3 files

Install software: 

sudo aptitude install poc-streamer

Cut file: 

mp3cut -o output.mp3 -t 00:00:20+000-00:00:58+000 input.mp3

More info:

[edit]

id3 manipulation (mp3 tags)

[edit]

Convert id3v1 tags

Some applications only read id3v2 tags, even though the mp3 file might have id3v1 tags as well. There is a nice utility which converts id3v1 tags to id3v2 tags. A very good feature in this application is that if the tag exist both in id3v1 and id3v2, id3v2 is not overwritten.

Use this command to convert id3v1 tags to id3v2 for all mp3 files in current directory and subdirectories. 

find -type f -iname "*.mp3" -exec id3v2 -C {} \;

iname means that it will be a case insensitive search, e.g. both .mp3 and .Mp3 files will be found.

After conversion we might as well remove the id3v1 tags: 

find -type f -iname "*.mp3" -exec id3v2 -s {} \;
[edit]

Set album artist

To make sure that albums with different artists is shown as a single album in your multimedia application, it is a good idea to set the Album artist tag (the tag might also be called band, orchestra or accompaniment) to Various artists. The code for the tag is TPE2, if you recognize this it might be because the Artist tag's code is TPE1.

The command to to tag your files is: 

id3v2 --TPE2 "Various artists" *.Mp3

To tag all your albums with one command:

  1. Go to the directory containing all your "Various artists"-albums
  2. run: 
    find -type f -iname *.mp3 -exec id3v2 --TPE2 "Various artists" {} \;


[edit]

Generate filename from id3 info (mp3 tags) 

id3 -f "%a - %t.mp3" *
[edit]

Generating id3 tags from filename and directory

Download the program id3 from here. Note: It is not the same as if you would run sudo aptitude install id3, as that is a completely different (and not as competent) program.

Let's start with an example: 

id3 -2 -lnat %1 %2 %3 %4  "Beethoven - */* * - *.mp3"

-lnat %1 %2 %3 %4 tells the program to parse

  • the first star as the album name (l)
  • the second star as the track number (n)
  • the third star as the artist name (a)
  • the fourth star as the song title (t)

This will for example tag the file: 

Beethoven - Concert No 1/01 Beethoven - Play 1.mp3

to: 

Song Name: Play 1
Artist: Beethoven
Album: Concert No 1
Track: 01

Run 

man id3

or check the webpage http://home.wanadoo.nl/squell/id3.html for more info.

[edit]

Tagging script

I have a created a neat script to speedup the process of tagging an album. Go get it here. Read the comments to understand how it works!

[edit]

Command-line-fu

"The best linux commands on the web" - http://www.commandlinefu.com/commands/browse/sort-by-votes

[edit]

Only download file it doesn't exist fully

Use the -c argument, e.g.: 

wget -c <url_to_file>

-c actually tries to continue downloading a file, thus it will not do anything if it's already completely downloaded.

[edit]

Comparing directories 

diff -qr dirA dirB | sort

The sort thing is optional, of course.

[edit]

Setting file owner and group in one command 

chown <username>:<usergroup> <file>
[edit]

Sending output to file

Quick guide:

  • Stdout - >
  • Stderr - 2>
  • both - &>

For a more descriptive guide, have a look here.

[edit]

Generate audio (mp3) playlist

  1. Install Fast Audio Playlist Generator (package fapg)
  2. Goto directory with files
  3. Run fapg --output=list.m3u .
[edit]

Check system information

CPU: 

cat /proc/cpuinfo

Memory: 

cat /proc/meminfo
[edit]

Remove empty directories

Run the following command: 

perl -MFile::Find -e"finddepth(sub{rmdir},'.')"

Thanks to Andrew Hardwick

[edit]

Search by description in Aptitude

The command line package manager aptitude allows you to search for packages by using: 

sudo aptitude search <search_string>

This will, however, only search for the string inside the package name. To search in the package description instead use: 

sudo aptitude search ~d<search_string>

An example: 

sudo aptitude search ~dthunderbird

Will list all packages that has the string thunderbird in its description.

[edit]

Shell scripting

Some hints when writing shell scripts. For a better understanding of how it works, check out a collection of my scripts here. To learn even more, check out the excellent Linux Shell Scripting Tutorial!

[edit]

Extracting parts of text

Use sed with regexp. Example: 

current="hello./Joe"
extract=$(echo "$current" | sed "s/.*\(\.\/.*\)/\1/")
echo $extract

Will give the result: 

./Joe
[edit]

Conditionals with AND and OR

AND 

if [ expr ] && [ expr ]
...

OR 

if [ expr ] || [ expr ]
...

Thanks to: http://www.injunea.demon.co.uk/pages/page205.htm

[edit]

Check if file exists 

if [ -e file ]
then
   such and such
fi

Also works for directories.

Thanks to: http://linuxreviews.org/beginner/bash_GNU_Bourne-Again_SHell_Reference/

[edit]

The IFS variable

The IFS (Internal Field Separator) variable determines how to delimit a string when looping through items in the string.

An example of when it could be useful to change this variable:

If you want to loop through files and directories (that might contain spaces) you don't want to have space as a delimiter. By default though, space is one of the characters in IFS variable. By setting: 

IFS=""

space will no longer be a delimiter, instead only newline will be.

Code example: 

IFS=""
for line in `find`
do 
   # Print the directory
   echo "$line"
done
IFS=$OIFS #Restore IFS

Without the IFS modification, files containing spaces would be printed on multiple rows, each row containing one word. Not what we want, right?

Thanks to:

[edit]

Save output of a command 

cnt=$(expr $cnt + 1)

This example saves the result of expr $cnt + 1 to $cnt. I.e. it increases $cnt by one.

[edit]

Unset variables 

unset var

Almost too easy, right?

[edit]

Get input from user 

read -p "Press enter to continue..."

Does not care about the input, pretty much just a pause. 

read savedInput

Stores what the user writes to variable $savedInput.

[edit]

Test if string matches a certain pattern 

  number=$(echo "$tagPattern" | egrep -x "[0-9]*")
  if [ $number ]
  then
     echo "It's a number, and the number is: $number"
  else
     echo "Not a number"
  fi
  • egrep - extended regular expressions support (same as grep -e)
  • -x - Entire string must match
[edit]

Arrays

[edit]

Fill array 

  while read line
  do
     patterns[${#patterns[*]}]=$line
  done < /home/user/myfile

Fills pattern array, each item in the array containing a line from the file.

[edit]

Loop through array 

  for (( i = 0 ; i < ${#patterns[@]} ; i++ ))
  do
     echo "$i. ${patterns[$i]}"
  done

Loops through the array and prints its content.

[edit]
Example

Download a series of images, in this example a number comic strips: 

for ((i = 7;i<10;i++)); do wget http://www.dn.se/content/1/c6/35/75/31/roc_2008060$i.gif -P /home/cewan/www/rocky; done
[edit]

Copy content from CD/DVD in a neat way 

eject -t && 
 mount /media/cdrom0 && 
 cp -Rfv /cdrom/* . && 
 eject -r

The previous row does the following:

  1. Closes the cd tray
  2. Mounts the disc
  3. Copies all the content of the disc to the current directory ((R)ecursively, (f)orce overwrite, (v)erbose)
  4. Opens the tray again
[edit]

Image manipulation

[edit]

Resize images 

mogrify -sample 800x800 *

Will resize images to 800x800 (keeping the aspect ratio). NOTE that it will overwrite the original.

[edit]

Generating image file names from EXIF info

Install package renrot. Then goto the folder with your files and type: 

renrot -n "%Y-%m-%d_%H-%M-%S_%c" *

This will generate filenames looking like: 

2007-05-13_01-55-_127.JPG
[edit]

Modifying EXIF information

Use the exiv2 program. Example: 

exiv2 -a 1 *JPG

Adjusts the EXIF timestamp one hour ahead for all files ending with JPG.

Another example: 

exiv2 pr *

Prints EXIF information for all files.

[edit]

Network

[edit]

Listening ports 

nmap -sT -O localhost

OR 

netstat -tulnap
[edit]

Bandwidth usage

Nethogs is an amazing program which display bandwidth usage per process. Simply install package nethogs and you are good to go. 

sudo nethogs

If you have more than one network adapter you might want to tell nethogs which adapter to monitor, eth0 is the default. 

sudo nethogs eth1
[edit]

Locate

The locate command can be used to find any file quickly, as long as it is indexed in its database.

Example: 

locate --regex -i veronica.+maggio.+barn

Case insensitive (-i) search using regular expressions (--regex). The ".+" used here is the wildcard in the regex world. For instance the following file would match this search: 

/media/Arkiv2/Mp3/Music/Singles/2008-12/Veronica Maggio - Måndagsbarn.mp3


Locate is configured by editing the file /etc/updatedb.conf, where amongst other things one can set what folders to include (or exclude) when indexing. When the configuration has been modified it is usually a good idea to update the index database, by running updatedb.

[edit]

Midnight Commander

midnightCommander.jpg

The terminal is nice and powerful, but what if you, for instance, want to easily select a range of files for moving to another directory? A file manager would be nice for this.

Luckily, there exists a nice and powerful console-based file manager called Midnight Commander. The package that must be installed is mc, and the program is run by writing mc.

[edit]
Good to know
  • Individual files/folders can be selected with the 'Insert' button
  • Jump to file - press Ctrl-S (or Alt-S)
[edit]
Links
[edit]

Searching

[edit]

Searching on content

[edit]

Search on all files 

grep -r <search_string> * .*

-r means that it will be a recursive search

'*' means that it will search all files and folders (except hidden ones)

'.*' means that it will search all hidden files and folders

Example: 

grep -r eclipse * .*

will find return all lines containing eclipse.

[edit]

Search on files matching a specific name pattern 

find <folder_name> -name *.xml -exec grep -i apple {} \; -print

This will search the folder <folder_name> recursively, from the current directory (.), only in files having the extension xml (*.xml), having the content apple, case insensitive (-i), and finally print the filename that has this content (-print)

[edit]

Search and replace in multiple files

[edit]
Alternative 1

  1. Create a new file: vim replace.sh
  2. Add the following content: 
    for fl in *.php; do
     perl -p0777i -e 's/FINDSTRING/REPLACESTRING/g' $fl
     done
  3. Replace *.php with your own file pattern, and replace FINDSTRING and REPLACESTRING.
  4. Run the script in desired folder!
The reason we are not using sed instead perl is that sed has some issues with the newline character.
[edit]
Alternative 2

Create a script with the following content: 

  echo "Text to be replaced:"
  read replaced
  echo "Text that will be replacing:"
  read replacing
  find . -name *.xml -exec perl -p0777i.bak -e "s/$replaced/$replacing/g" {} \;

This script will let the user enter what text to replace, and what text will be replacing that. The operation will only be performed on files with extension xml (*.xml). '-p' will print the result, -0777 will treat the content of the file as a single line, -i.bak will write to file, and create a backup with the extension bak.

NOTE: Make sure that you use double quotes (") here, and not single quotes('). Otherwise parameterpassing will not work, i.e. the dollar sign ($) will be treated as just text.

[edit]

Search on filename

[edit]

Finding files

To find files with specific name patterns, type for example: 

find /usr -name *spec

This will find every file under the directory /usr ending in ".spec".

The -name option is somewhat limited, for a more powerful search use regular expressions: 

find . -regex '.*heme.*'

The search will find all files having heme somewhere in its name.

Thanks to Linux.ie's guide!

[edit]

Rename multiple files (recursively)

[edit]
Alternative 1

For all filenames ending with .avi, replace 'find' with 'replace': 

for i in *.avi; do j=`echo $i | sed 's/find/replace/g'`; mv "$i" "$j"; done
[edit]
Alternative 2

To rename all files and folders in current directory: 

rename 's/find/replace/g' *.avi

To make it recursive: 

find . -exec rename 's/find/replace/g' *

Thanks to TylerHall and Mind.

[edit]
Rename using regular expressions

For every file in the current directory, rename all occurences of a digit followed by a space TO the same digit followed by " Radiohead - ": 

rename 's/(\d) /$1 Radiohead - /g' *

Example: 

01 Airbag.mp3

will be renamed to: 

01 Radiohead - Airbag.mp3

Another example 

rename 's/(.*) - (.*).mp3/$2 - $1.mp3/g' *

move beginning of filename to the end and vice versa. Ie: 

Use Somebody - Kings of Leon.mp3

Will become: 

Kings of Leon - Use Somebody.mp3

For more about regular expressions see: Regular Expressions

[edit]

Find files (and not directories)

To find files only, in the current directory and not in subdirectories: 

find . -maxdepth 1 -type f
[edit]

Searching inside zip (and jar) files

Problem You want to:

  • locate a file inside a zip file
  • search inside multiple zip files
  • search recursively on all zip files located in all subfolders from the current folder

Solution This little script I wrote will help you, you can also download it here: 

#!/bin/bash
#
# Author: Johan Frick, http://cjf.se, 2007
if [ $# -ne 2 ]
then
 echo "Usage: zipgrepr <(part of) filename to search for> <file extension>"
 echo "Example: To search for the file MyClass.class inside jar files: 'zipgrepr MyClass jar'"
else
 find . -type f -name "*" | grep "$2" > TmpMatches

 while read current
 do
        match=$(jar tvf "$current" | grep $1)
        if test -n "$match"
        then
        echo -e "-=$current=- \n $match"
        fi
 done < TmpMatches
fi

As you can see I am using the jar tool to search inside each jar/zip file. This works as jar files are zip files with just the extension changed.

Thanks go to Vivek G. Gite who wrote the excellent hand book on shell scripting!

[edit]

Clearing terminal window 

clear

or simply press Ctrl+L

[edit]

Saving your password for ssh connections

This tutorial will let you store the password for an ssh connection so that you don't have to write the password every time you login!

  1. Create a public/private keypair using: 
    ssh-keygen -t rsa

    Press enter on the questions that come up, to make sure the stuff is saved at the default location and that you do not have to enter a password when loggin in to your ssh server.

  2. Copy the public key to the server using scp (note the ending colon after the server hostname, it tells the path): 
    scp ~/.ssh/id_rsa.pub username@server:
[enter password]

    3. Note the colon (:) after the server name!

  3. Log into the server and create .ssh dir if it doesn't exist, then add the key: 
    ssh username@server
[enter password]
mkdir .ssh (if it doesn't already exist)
cat id_rsa.pub >> .ssh/authorized_keys
  4. Now you're done! Try logging out and then back in to verify you don't need the password anymore: 
    press ctrl+d
ssh username@server

More info here.

[edit]

Debian packages

To install debian packages (*.deb): 

dkpg -i <filename>


[edit]

Archiving

[edit]

Extracting from command line

To extract a compressed file (*.tar.gz): 

tar  -xvf file
[edit]

Creating archive

tar -pczf name_of_your_archive.tar.gz /path/to/directory where:

  • p - preserve permissions
  • c - create archive
  • z - use gzip to compress
  • f - use the following archive name

see tar --help for more info.

[edit]

Aliases (kind of like Windows shortcuts)

There are commands that you type quite frequently in Linux that would be nice to assign to something more short-typed. To assign edit your .bashrc-file by writing: 

vim ~/.bashrc

Vim is a very common editor in Linux. At first it might seem awkward but once you get the hang of it it is actually quite nice! Go here to learn how to use it. If you have another favorite editor you can of course use that one instead.

Here are some nice to have aliases: 

alias ls='ls --color=auto'
alias l='ls -la'
alias s='cd ..'
alias srcBashrc='source ~/.bashrc'
alias nBashrc='nano ~/.bashrc'
alias top='top -c -d 1'

Add them wherever you like in your .bashrc-file.


[edit]

Counting files

To count the number of files type: 

find . -type f | wc -l

Works recursively from the current (.) directory.

[edit]

Size of folder 

du -ksh .

Works recursively on the current (.) directory.

[edit]

Size of all immediate subfolders 

find -maxdepth 1 -type d -exec du -ksh {} \;
[edit]

Screen

You have logged in to your linux box using ssh and you are currently running a big operation when suddenly the connection is lost. Ouch, now we have to do it all over again. To avoid this pain use the screen function. Screen is something that will run even if you are not currently logged in. It's very neat, and it's easy to use. Read exactly how here!

[edit]

Scrolling

To be able to scroll in the page buffer you would normally use shift-page up/down, but this doesn't work in screen mode. By pressing Ctrl+A then ESC you enable scroll mode and you can now scroll up/down by pressing page up/down.

Thanks to http://gentoo-wiki.com/TIP_Using_screen#Scrolling_back_in_Screen

[edit]

Update a folder

Almost too easy to write, but still so very useful. If you want to keep a folder synchronized with another folder simply use 

cp -ur <source-folder> <dest-folder>

u means to only copy files that are newer or does not exist in destination. r means recursively.

[edit]

Deleting a specific file/folder from multiple locations

...recursively 

find . -name <name of file/folder> -exec rm -rf {} \;
[edit]

Top alternative

Top is a command line tool for monitoring resource usage. There exists an enhancement of this tool called htop, which, as its creators say, is an interactive process viewer. Try it out by installing the package 

htop

and then running 

htop
[edit]

vim

Some hints for the editor Vim. More can be found in the Vim manual.

[edit]

Auto commenting

By default vim automatically adds comment signs when adding a newline on a commented line. This can be annoying when pasting chunks of text. To disable it add the following to vimrc: 

au FileType * setl fo-=cro
[edit]

Regular expressions

Regexps are very powerful when, for instance, searching for and replacing text. Check out this tutorial to learn how to use it in vim.

[edit]

Spaces instead of tabs

Use appropriate lines from 'Configuration' section below. If your file is already full of annoying tab characters, run the command: 

:retab
[edit]

Autoindent

Use appropriate lines from 'Configuration' section below.

Now open your the file that you want indented in vim. The press the following 

gg=G

Nice, huh?

[edit]

Counting words, lines, bytes etc

While in command mode (ESC): 

g Ctrl-g
[edit]

Sorting 

:%!sort
[edit]

Line numbers 

:set number
:set nonumber
[edit]

Browse through previous commands

  1. Type ':'
  2. Use up and down arrows
[edit]

Cut and paste 

dd

to cut line 

P

or 

p

to paste

[edit]

Selecting text (Visual mode)

  1. Use 'v' to enter select mode
  2. Move arrows to select
  3. Use 'y' to copy, or 'd' to cut
  4. 'p' will paste the stuff

Check out the Vim manual for more on visual mode.

[edit]
More selection options

If you instead press:

  • 'V' you will enter select mode and select whole lines
  • Ctrl+v you will enter select mode and select using a box
[edit]

Search and replace

find each occurance of 'foo' and replace it with 'bar' without asking for confirmation: 

:%s/foo/bar/g
[edit]

Configuration file

Make sure that the package vim is installed, and not only vim-common. Then edit your /etc/vim/vimrc as follows (usually the lines are there, just uncomment them): 

runtime! debian.vim

" Jump to the last position when reopening a file
if has("autocmd")
  au BufReadPost * if line("'\"") > 0 && line("'\"") <= line("$")
    \| exe "normal g'\"" | endif
endif

" Load indentation rules according to the detected filetype
if has("autocmd")
   filetype indent on
endif

syntax on            " syntax highlighting
set background=dark  " dark background
set autoindent       " Automatically indent
set tabstop=3        " Number of spaces that a tab counts for
set shiftwidth=3     " Number of spaces to use for each step of (auto)indent
set expandtab        " Use spaces instead of tabs
set number           " Line numbering
set showmatch        " Show matching brackets.
set ignorecase       " Do case insensitive matching
set smartcase        " Do smart case matching, i.e. case sensitive if upper case is used
set incsearch        " Incremental search

au FileType * setl fo-=cro
Hämtad från "http://cjf.se/wiki/index.php/Working_with_the_terminal"
Software