Whats new in JDK 7

Below are some new features in JDK 7

• NIO 2.0 The java.nio.file package and its related package, java.nio.file.attribute, provide comprehensive support for file I/O and for accessing the file system.
• Sockets Direct Protocol Sockets Direct Protocol (SDP) is a wire protocol developed to support stream connections over InfiniBand fabric.
• Compressed 64-bit object pointers. A technique for compressing 64-bit pointers to fit into 32 bits, which decreases memory and memory-bandwidth consumption and thereby improves performance
• VM support for non-Java languages. VM extensions to support the implementation of non-Java languages at performance levels near to that of the Java language itself.
• Garbage-First GC. A new garbage collector that promises to achieve lower pause times and better predictability than the current CMS collector

Determine files that are modified

find . -mtime -1 -print

Using Awk to rename multiple files

The following awk script will rename files named like:

_TEST.ABC.123.XML.1231123

to the following

TEST.ABC.123.XML


for f in `ls _TEST*`
do
newfile=`echo $f | grep -o TEST.[A-Z]*.[0-9]*.XML`
mv $f $newfile
done

Encrypting file on Unix

crypt: Encodes (encrypts) or decodes a file
usage: crypt [options][file]
options: password
example: crypt key <> encrypted.file
example: crypt key < encrypted.file | lpr

Resource:

http://www.u.arizona.edu/udocs/unix-cmds.html#crypt

Moving a foreground process to the background in UNIX

Below is an example of moving a process already running in the foreground to the background:

1) To start we have a script running in the foreground

/home/javaoneman>runTest.sh

2) Press CTRL-Z to place the process in a suspended state.

The following should appear on the console:

^Z

[1]+ Stopped runTest.sh

3) Move the suspended process into the background using the bg command

/home/javaoneman>bg

The following confirmation should appear on the console:

[1]+ runTest.sh &

Thanks to Dalhousie University Computer Science Department for providing this useful information on UNIX process control

Using sed to remove lines from multiple files

In addition to searching and replacing content in files, Sed can also remove specific characters from files. This can be done by specifying the '/d' option after the pattern you specific. Below is a script which deletes the DTD header for a list of files in a specific directory using the Sed command:


##########################################################
#
# Script used to delete the DOCTYPE header consisting of
# the dtd file path from each xml in a specified directory
#
##########################################################

INPUT_DIR=$1

for filename in $INPUT_DIR/*.xml
do
echo "removing dtd path for $filename ...."
sed -e '/DOCTYPE/d' $filename > $INPUT_DIR/temp_file
mv -f $INPUT_DIR/temp_file $filename
done

Unix's find command

Introduction

The find command allows the Unix user to process a set of files and/or directories in a file subtree.

You can specify the following:

• where to search (pathname)
• what type of file to search for (-type: directories, data files, links)
• how to process the files (-exec: run a process against a selected file)
• the name of the file(s) (-name)
• perform logical operations on selections (-o and -a)

Search for file with a specific name in a set of files (-name)

find . -name "temp.txt" -print

This command will search in the current directory and all sub directories for a file named temp.txt.

Note: The -print option will print out the path of any file that is found with that name. In general -print wil print out the path of any file that meets the find criteria.

How to apply a unix command to a set of file (-exec).

find . -name "temp.txt" -exec chmod o+r '{}' \;

This command will search in the current directory and all sub directories. All files named temp.txt will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended.

The end results of this command is all rc.conf files have the other permissions set to read access (if the operator is the owner of the file).

How to apply a complex selection of files (-o and -a).

find /usr/tmp -not \( -name "*,v" -o -name ".*,v" \) '{}' \; -print

This command will search in the /usr/tmp directory and all sub directories. All files that are of the form '*,v' and '.*,v' are excluded. Important arguments to note are:

• -not means the negation of the expression that follows
• \( means the start of a complex expression.
• \) means the end of a complex expression.
• -o means a logical or of a complex expression.
  In this case the complex expression is all files like '*,v' or '.*,v'

The above example is shows how to select all file that are not part of the RCS system. This is important when you want go through a source tree and modify all the source files... but ... you don't want to affect the RCS version control files.

How to search for a string in a selection of files (-exec grep ...).

find . -exec grep "foo" '{}' \; -print

This command will search in the current directory and all sub directories. All files that contain the string will have their path printed to standard output.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q "foo" '{}' \; -print

This command is very important for process a series of files that contain a specific string. You can then process each file appropriately.

Using lsof

The lsof (stands for list of open files) provides information on file and ports which are opened by any process currently running. Below is an example:

# lsof -i -nP | grep httpd
httpd 2318 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)
httpd 2319 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)
httpd 2322 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)

Example of using Sed

SED is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ED), SED works by making only one pass over the input(s), and is consequently more efficient. But it is SED's ability to filter text in a pipeline which particularly distinguishes it from other types of editors.

One of sed's most useful commands is the substitution command. Using it, we can replace a particular string or matched regular expression with another string. For example say we have a file (called test.txt) with the following content:

aaa bbb ccc

eee fff ggg

bbb ccc aaa

eee rrr kkk

We want to change all the instances of “aaa” to “zzz”. The following sed command could be used to perform the search and replace:

sed -e 's/aaa/zzz/g' test.txt > out.txt

The contents of the output file (out.txt) now has the following:

zzz bbb ccc

eee fff ggg

bbb ccc zzz

eee rrr kkk

Unix shell script to automate FTP process

I found the following script below from FTP Planet to automate FTP process. It seems to work well but do not recommend leaving your username and password in any script for a extended period of time:

ftp -v -n hostname << EOF
user usuario password
bin
put filename
bye
EOF

Removing carriage returns (^M) from a file

When transferring a file from UNIX to Windows in binary mode, you may observe carriage returns (control M characters) in the file when viewing it on UNIX. The control M character(^M) is the end of line character on Windows . When you transfer afile via FTP in binary mode, the file is transferred with the carriage returns intact. The control M character is not the end of line character in UNIX, so as a result the ^M is interpreted as part of the file. In order to avoid this problem the file should be transferred in ASCII mode. If this is not possible there are scripts which can remove the Control M character. Below is an example of using the tr command to remove the control M characters from a file:

tr -d "\015" < $input > $output

The tr command reads in characters from standard input and replaces or deletes characters before writing them back to standard output. The -d option in the example above will delete the carriage returns (specified in octal as o15) from the specified input file ($input) and redirect the results to an output file ($output). Here is in a for loop:

for f in `ls `

do

tr -d "\015" <$f >tmp/$f

done