Squashing spaces

If we want to remove extra spaces in a text like this:

You could be tempted to use sed, but there is a simpler way using tr

And we end up with:

With tr you can also delete characters instead of compressing them. Let’s say we wanted to get rid of the vowels.

We get:

Variables with Awk

Awk comes with some predefined variables, like NF for number of fields or OFS for field separator. If you wanna know more about these ‘man awk’ and search for ‘Built-in Variables’. In this post we are going to talk about using your own variables just like in any other programming language. If you are new to awk start here: http://www.blackbytes.info/2012/01/intro-to-awk/

Awk variables example

If we want to find the biggest number in a file we could do something like this:

There is a few interesting things in this line. To start with we don’t need to declare our variable which is cool, but what I want you to pay attention to is that variables in awk don’t have a leading $, neither when assigning or accessing the value, this may be a bit confusing if you are used to do bash scripting. Here the leading $ always references fields so if you used $max instead it wouldn’t work as expected.

Then there is this END statement, which allows us to execute some code after all the lines of input have been processed. There is also a BEGIN statement which does exactly what you would expect.

Finally here is an Awk reference card you may find useful: http://www.catonmat.net/download/awk.cheat.sheet.pdf

Mass renaming

Say you have a bunch of files that you want to rename, there is 2 easy approaches to this problem: you can use the aptly named ‘rename‘ utility which should be available on most Linux or you can use a FOR loop. Let’s explore see both ways of mass renaming.

Our job is to rename these images to have a lowercase extension:

Using the rename tool

We can achieve this with the rename command:

Problem solved, easy as that! As you can see the syntax is similar to sed. Basically we are saying “substitute .JPG for .jpg for all files in this directory”

Using a for loop

Now using a for loop we are going to convert these extensions back to uppercase.

There is also a slight variation:

Beyond Bash

If you use Linux you are probably using the bash shell, but turns out there is life beyond bash!

When using aptitude or apt-get I’m sure you tried this at least once:

aptitude ins<tab>
"Oops, that didn't work but I wish it did!"

Getting started with ZSH

Well that and much more is possible using another shell: ZSH, to get started you will want to install it (should be available in your distro repo) . After that just run ‘zsh’ you may get a dialog asking if you would like to create a basic configuration file (.zshrc just like .bashrc) just skip it but don’t get scared when your prompt looks like this:

[e]0;u@h: wa]${debian_chroot:+($debian_chroot)}u@h:w$

This is happening because it’s trying to use your bash prompt which uses a different syntax, now we are going to install “oh-my-zsh” which contains a decent default config, various plugins and it will set zsh as your default shell (you can revert using chsh -s /bin/bash user) also it comes with themes which are nothing more than prompt configurations, you can see images on how they look here:

https://github.com/robbyrussell/oh-my-zsh/wiki/themes

To install oh-my-zsh issue this command:

Now we are ready to start playing with our new shell, let’s explore some of the features.

Better Tab completion

Zsh is able to autocomplete the arguments for most system commands, and even give us a menu-like interface if you tab twice.

This also works with the kill command:

Typo correction

Zsh will help you to fix those annoying typos you make all the time:

Auto push

If you jump a lot between different dirs you are going to love this, zsh can autopush the dirs you cd into, this means you will be able to list the recent dirs you have been in using the ‘d’ command (which is an alias for dirs -v) and then change to them just typing the corresponding number.

And that’s just a small taste of what zsh has to offer, now it’s your turn to try it and decide if you like it.

Zsh FAQ: http://zsh.sourceforge.net/FAQ/
Zsh documentation: http://zsh.sourceforge.net/Doc/

CLI ninja: Ping Sweep

Ever wanted to do a ping sweep in this new network you just broke in but you don’t want (or can’t, for some reason, AVs etc…) to upload any tools? Or even in your own network but you don’t have time to install nmap for whatever reason?

Well, you can still do it by leveraging the OS built-in tools. With a for loop we can launch a ping for a whole class C in about 3 min in windows and in about 10 sec in Linux.

Linux Ping Sweep

for i in {1..254} ;do (ping -c 1 192.168.1.$i | grep "bytes from" &) ;done

What this does is a for loop from 1 to 254, $i takes the value of the current iteration so in the first one it will be 1 then 2, 3… and so on, then we tell it to call the ping command with the -c option which means only ping once otherwise it would ping forever after that we pipe the output to grep so we only see the hosts that actually responded and the & at the end send it to the background so it will launch all the pings in parallel. If we only want the ip address and not the whole line we can further filter this using cut.

Windows Ping Sweep

for /L %i in (1,1,255) do @ping -n 1 -w 200 192.168.1.%i > nul && echo 192.168.1.%i is up.

As you can see the idea is the same, -n being the equivalent of -c in Linux’s ping and -w is the timeout, then we send the output to nul and echo only if the ping command was successful (that’s what the && is for)