AWK去除某列或某行数据

一、去列

有些情况下, 我们再文本处理的时候需要把前面几列删除掉,特别是在使用去重uniq命令的时候。

使用AWK可以解决问题。

例如,一个文件file.txt内容如下,需要进一步的去重动作,此时需要删除第一列再使用uniq命令。

 

     13   app_version: '1' } {}
     17   app_version: '1' } {}
      1   app_version: '1' } {}
      1   app_version: '1' } {}
      1   app_version":"1.2.0","
      1   app_version":"1.2.0","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version":"1.2.1","
      1   app_version=1.2.1&app_
      2   app_version: '1' } {}
      2   app_version: '1' } {}
      3   app_version: '1' } {}
      3   app_version: '1' } {}
      3   app_version: '1' } {}
      4   app_version: '1' } {}
      4   app_version: '1' } {}
      4   app_version: '1' } {}
      7   app_version: '1' } {}

使用AWK运行结果如下,

[pc@centos603 ~]$ awk '{ $1="" ; print $0 }' file | uniq -c | sort
     11  app_version":"1.2.1","
      1  app_version=1.2.1&app_
      2  app_version":"1.2.0","
      4  app_version: '1' } {}
      9  app_version: '1' } {}

二、去行

也有些情况下,我们要删除错误输出、警告输入或末尾的行,以便进一步统计,来个例子。

在用命令netstat -ntu 2>/dev/null 统计网络连接时,输出如下,

[pc@centos603 ~]$ netstat -ntu  2>/dev/null
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 10.10.18.79:22              10.10.18.52:53752           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53753           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:26733           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53751           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:31682           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:40470           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.95:49174           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53755           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.219:63830          ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53756           ESTABLISHED

 

前面两行可能不是我们想要的,怎么去除 ??答案如下,

[pc@centos603 ~]$ netstat -ntu  2>/dev/null | awk 'NR != 1 && NR != 2'
tcp        0      0 10.10.18.79:22              10.10.18.52:53752           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53753           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:26733           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53751           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:31682           ESTABLISHED
tcp        0    100 10.10.18.79:22              10.10.18.44:40470           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.95:49174           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53755           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.219:63830          ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53756           ESTABLISHED

 

更直接的方法:

[pc@centos603 ~]$ netstat -ntu  2>/dev/null | awk 'NR > 2'
tcp        0      0 10.10.18.79:22              10.10.18.52:53752           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53753           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:26733           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53751           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.44:31682           ESTABLISHED
tcp        0     36 10.10.18.79:22              10.10.18.44:40470           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.95:49174           ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53755           ESTABLISHED
tcp        0      0 10.10.18.79:445             10.10.18.219:63830          ESTABLISHED
tcp        0      0 10.10.18.79:22              10.10.18.52:53756           ESTABLISHED

 

 

 

 

 

 

 

 

发表评论

电子邮件地址不会被公开。