Wednesday, November 07, 2007

Remove First 10 or 100 or 1000 Lines in a Log File - UNIX

After reviewing my Google Webmaster tools, I've observed several search iterations for "remove first 100 or 1000 lines in a log file." Here's an example to demonstrate removing the first 10 lines, 100 lines or 1000 lines in an .htaccess file.

Remove first 10 lines
# nl -ba .htaccess | more
1
2 order allow,deny
3 allow from all
4

5
6 deny from 213.xxx.xxx.xx
7 deny from 124.xxx.xxx.xx
8 deny from 81.xxx.xxx.xx
9 deny from 88.xxx.xxx.xx
10 deny from 88.xxx.xxx.11
11 deny from 84.xxx.xxx.56
12 deny from 88.xxx.xxx.xx
13 deny from 82.xxx.xxx.xx
14 deny from 69.xxx.xxx.xx
15 deny from 24.xxx.xxx.xx
...

# tail +11 .htaccess | more
deny from 84.xxx.xxx.56
deny from 88.xxx.xxx.xx
deny from 82.xxx.xxx.xx
deny from 69.xxx.xxx.xx
deny from 24.xxx.xxx.xx
...

Above is for illustration purposes but this syntax is all you need.
# tail +11 .htaccess > remove_10Lines.txt

Remove first 100 lines
# nl -ba .htaccess | more
...
96 deny from 85.xxx.xxx.xx
97 deny from 85.xxx.xxx.xx
98 deny from 85.xxx.xxx.xx
99 deny from 83.xxx.xxx.xx
100 deny from 81.xxx.xxx.70
101 deny from 81.xxx.xxx.29
102 deny from 81.xxx.xxx.xx
103 deny from 77.xxx.xxx.xx
...

# tail +101 .htaccess | more
deny from 81.xxx.xxx.29
deny from 81.xxx.xxx.xx
deny from 77.xxx.xxx.xx
...

Above is for illustration purposes but this syntax is all you need.
# tail +101 .htaccess > remove_100Lines.txt

Remove first 1000 lines
# nl -ba .htaccess | grep 1000 | more
1000 deny from 88.xxx.xxx.114

# nl -ba .htaccess | more
...
996 deny from 88.xxx.xxx.xxx
997 deny from 88.xxx.xxx.xxx
998 deny from 88.xxx.xxx.xxx
999 deny from 88.xxx.xxx.xxx
1000 deny from 88.xxx.xxx.114
1001 deny from 88.xxx.xxx.59
1002 deny from 88.xxx.xxx.xxx
1003 deny from 88.xxx.xxx.xxx
...

# tail +1001 .htaccess | more
deny from 88.xxx.xxx.59
deny from 88.xxx.xxx.xxx
deny from 88.xxx.xxx.xxx
...

Above is for illustration purposes but this syntax is all you need.
# tail +1001 .htaccess > remove_1000Lines.txt

2 comments:

UX-admin said...

awk 'NR >= 1000 {print;}' filename.txt

esofthub said...

Thanks ux-admin for the tip