Catalog

1.sed brief introduction

2.sed Of p command -- Print matching lines

3.sed Of d command -- Delete specified row

4.sed Of a command -- Append... After the matching line

5.sed Of i command -- Insert... Before the matching line

6.sed Of c command -- Full line replacement

7.sed Of r command -- Read the contents of the file into

8.sed Of w command -- Write text to file

9.sed Of s command -- String substitution （ Match regular expression ） Core usage

One .sed command

1.sed brief introduction

sed Is a regular expression support Non interactive flow editor , It is the best tool for modifying or replacing text in scripts

sed Itself is also a pipeline command , It can replace data 、 Delete 、 newly added 、 Select specific lines and other functions

# Grammar format
sed   [ Options ]  sed Edit command   Input file

# Common options
-n： Show only the rows that match the processing （ Otherwise, all... Will be output ）
-r： Support for extended regular expressions
-i： Modify directly in the file , Instead of output to the screen
-e： When executing multiple editing commands （ It's usually used ; Instead of ）
-f： Read the contents from the script file and execute （ The editing commands in the file are one per line , no need ; separate ）

# Common editing commands
p： Print matching lines           print
d： Delete specified row           delete
a： Append... After the matching line     append
i： Insert... Before the matching line  insert
c： Full line replacement
r： Read the contents of the file into  read
w： Write text to file     write
s： String substitution （ Match regular expression ）substitution
=: Output line number

### The above operations can be based on the line number , You can also match according to the pattern

2.sed Of p command -- Print matching lines

### Print page 1 That's ok 
[[email protected] ~]# sed -n '1p' /etc/passwd
root:x:0:0:root:/root:/bin/bash

### Print page 2 Go to the first place 4 That's ok 
[[email protected] ~]# sed -n '2,4p' /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
[[email protected] ~]# sed -n '2,+2p' /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin

### Print the last line 
[[email protected] ~]# sed -n '$p' /etc/passwd
cali123:x:1026:1028::/home/cali123:/bin/bash

### Print the first and last line 
[[email protected] ~]# sed -n '1p;$p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
cali123:x:1026:1028::/home/cali123:/bin/bash

### Don't print the 4 Line to last line , That is, print the... Page 1 Go to the first place 3 That's ok , And display the line number 
[[email protected] ~]# sed -n '4,$!{=;p}' /etc/passwd
1
root:x:0:0:root:/root:/bin/bash
2
bin:x:1:1:bin:/bin:/sbin/nologin
3
daemon:x:2:2:daemon:/sbin:/sbin/nologin

### Print matching adm Line to match sync That's ok 
[[email protected] ~]# sed -n '/adm/,/sync/p' /etc/passwd
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync

### Print page 1 That's ok , The first 3 That's ok , The first 5 That's ok 
[[email protected] ~]# sed -n '1p;3p;5p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

### Print page 3 Xing He 5 That's ok 
[[email protected] ~]# sed -n -e '3p' -e '5p' /etc/passwd
daemon:x:2:2:daemon:/sbin:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

###  Start with the first line , Output one line every two lines 
[[email protected] 0713]# cat /etc/passwd -n|sed -n '1~2p'    #2 Is the step value , Two outputs apart 
     1	root:x:0:0:root:/root:/bin/bash
     3	daemon:x:2:2:daemon:/sbin:/sbin/nologin
     5	lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
     7	shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
     9	mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
    11	games:x:12:100:games:/usr/games:/sbin/nologin
    13	nobody:x:99:99:Nobody:/:/sbin/nologin

### Match with s Beginning line , Match according to the regular pattern , There is no line number 
[[email protected] 0713]# sed -n '/^s/p' /etc/passwd    
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
sc:x:1000:1000:sc:/home/sc:/bin/bash
sc1:x:1001:1001::/home/sc1:/bin/bash

### No display /etc/ssh/sshd_config Invalid line inside , That is, do not display blank lines or # Beginning line 
[[email protected] 0713]# sed -rn '/^#|^$/!p' /etc/ssh/sshd_config     #｜ It's extended regular 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server

### Display with / The line at the end of the symbol , Need to use escape symbols 
[[email protected] 0713]# df -h|sed -n '/\/$/p'
/dev/mapper/centos-root   17G  3.9G   14G   23% /
[[email protected] 0713]# df -h|egrep  '/$'    # Usually use egrep lookup 
/dev/mapper/centos-root   17G  3.9G   14G   23% /

shell The variables in are passed to sed--》 The question of reference transmission --〉 Use double quotes , Then put curly braces outside the variable

[[email protected] 0713]# a=5
[[email protected] 0713]# b=8
[[email protected] 0713]# sed -n "${a}p;${b}p" /etc/passwd
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
halt:x:7:0:halt:/sbin:/sbin/halt
[[email protected] 0713]# 
[[email protected] 0713]# sg=sc
[[email protected] 0713]# sed -n "/${sg}/p" /etc/passwd
sc:x:1000:1000:sc:/home/sc:/bin/bash

3.sed Of d command -- Delete specified row

[[email protected] lianxi]# cat cali.txt
cali	
hunan	
yongz               
rose
jack


### The following operations are only displayed on the screen , The actual document has not been modified , To modify directly in the file , Need to add -i Options 
# Delete 2 To 4 That's ok 
[[email protected] lianxi]# sed '2,4d' cali.txt
cali	
jack

# Delete the string containing z The line of 
[[email protected] lianxi]# sed '/'z'/d' cali.txt
cali	
hunan	
rose
jack

# Delete a string that does not contain z The line of 
[[email protected] lianxi]# sed '/'z'/!d' cali.txt
yongz   

# Delete blank lines and comments             
[[email protected] lianxi]# sed -r '/^$|^#/d' cali.txt
cali	
hunan	
yongz               
rose
jack

# Delete blank lines and with # Beginning line 
[[email protected] 0713]# sed -i -r  '/^#|^$/d' sshd_config 
[[email protected] 0713]# cat sshd_config 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server
[[email protected] 0713]# 

4.sed Of a command -- Append... After the matching line

### The append operation can be performed according to the line number and pattern matching 
[[email protected] lianxi]# cat cali.txt
cali	
hunan	
yongz               
rose
jack

# In the 2 Add a string after the line 1000
[[email protected] lianxi]# sed '2a 1000' cali.txt
cali	
hunan	
1000
yongz               
rose
jack

# Include in root String is appended below the line of 1000
[[email protected] lianxi]# sed '/root/a 1000' /etc/passwd
root:x:0:0:root:/root:/bin/bash
1000
operator:x:11:0:operator:/root:/sbin/nologin
1000

5.sed Of i command -- Insert... Before the matching line

###  The insert operation can be performed according to the line number and pattern matching 
[[email protected] lianxi]# cat cali.txt
cali	
hunan	
yongz               
rose
jack

# stay cali.txt Insert a string before the last line of 1000
[[email protected] lianxi]# sed '$i 1000' cali.txt
cali	
hunan	
yongz               
rose
1000
jack

# Include in root Insert a string before the line of 1000
[[email protected] lianxi]# sed '/root/i 1000' /etc/passwd
1000
root:x:0:0:root:/root:/bin/bash
1000
operator:x:11:0:operator:/root:/sbin/nologin

6.sed Of c command -- Full line replacement

####  Changing the entire row can be performed according to the row number and pattern matching 

[[email protected] lianxi]# cat cali.txt
cali	
hunan	
yongz               
rose
jack

# Will be the first 3 Replace the whole line with a string 1000
[[email protected] lianxi]# sed '3c 1000' cali.txt
cali	
hunan	
1000
rose
jack

# take root Where the line is    Replace the whole line with a string 1000
[[email protected] lianxi]# sed '/root/c suda' /etc/passwd
suda
bin:x:1:1:bin:/bin:/sbin/nologin


[[email protected] 0713]# cat ifcfg-ens33 
BOOTPROTO="none"
NAME="ens33"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.2.43
PREFIX=24
GATEWAY=192.168.2.1
DNS1=114.114.114.114
[[email protected] 0713]# sed -i '4c ONBOOT="no"' ifcfg-ens33 
[[email protected] 0713]# cat ifcfg-ens33 
BOOTPROTO="none"
NAME="ens33"
DEVICE="ens33"
ONBOOT="no"
IPADDR=192.168.2.43
PREFIX=24
GATEWAY=192.168.2.1
DNS1=114.114.114.114
[[email protected] 0713]# sed -i '/^BOOTPROTO/c BOOTPOROTO="yes"' ifcfg-ens33 
[[email protected] 0713]# cat ifcfg-ens33 
BOOTPOROTO="yes"
NAME="ens33"
DEVICE="ens33"
ONBOOT="no"
IPADDR=192.168.2.43
PREFIX=24
GATEWAY=192.168.2.1
DNS1=114.114.114.114
[[email protected] 0713]# sed -i '/^ONBOOT/ s/no/yes/' ifcfg-ens33 
[[email protected] 0713]# cat ifcfg-ens33 
BOOTPOROTO="yes"
NAME="ens33"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.2.43
PREFIX=24
GATEWAY=192.168.2.1
DNS1=114.114.114.114

7.sed Of r command -- Read the contents of the file into

###  The read operation can be performed according to the line number and pattern matching 

#  stay /etc/passwd Read in after the end of the file cali.txt The content of the document 
[[email protected] lianxi]# sed '$r cali.txt' /etc/passwd
califeng:x:1025:1027::/home/califeng:/bin/bash
cali123:x:1026:1028::/home/cali123:/bin/bash
cali	
hunan	
yongz               
rose
jack

8.sed Of w command -- Write text to file

[[email protected] /]# cat selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

# hold selinux China and Israel # Beginning line , Don't write a.txt In file 
[[email protected] /]# sed '/^#/!w a.txt' selinux
[[email protected] /]# cat a.txt

SELINUX=disabled
SELINUXTYPE=targeted

9.sed Of s command -- String substitution （ Match regular expression ） Core usage

        1.sed Basic syntax of substitution ：

 sed 's/ Original string / Replaced string '

        2. Special character styles require escape symbols \, But escape characters cannot be used in single quotation marks , therefore sed The replacement syntax can also be written as ：

 sed "s/ Original string / Replaced string "

        3. You can add one at the end of the matching formula g, This will replace each matching string , Otherwise, only replace the first matching string in each line

 sed "s/ Original string / Replaced string /g"

        4. The use of some special characters ：

#  ^ What does it start with ,$ What does it end with 
#  Notice the `&` Symbol , without `&`, It will directly replace the matching string 
sed 's/^/ Added header &/g' 　　　　 # Add... At the beginning of all lines 
sed 's/$/& Added tail /g' 　　　　 # Add... At the end of all lines 
sed '2s/ Original string / Replace string /g'　 # Replacement section 2 That's ok 
sed '$s/ Original string / Replace string /g'   # Replace the last line 
sed '2,5s/ Original string / Replace string /g' # Replace 2 To 5 That's ok 
sed '2,$s/ Original string / Replace string /g' # Replace 2 To the last row 

[[email protected] 0713]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 


[[email protected] 0713]# sed -i '/^SELINUX=/ s/enforcing/disabled/' /etc/selinux/config 
[[email protected] 0713]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

  5. Multiple substitutions can be executed in the same command , Use ; Separate , Or use -e Options

#  Execute two replacement rules at the same time 
sed 's/^/ Added header &/g;s/$/& Added tail /g'
sed 's/^/ Added header &/g -e s/$/& Added tail /g'

6. Edit a test.txt file , The contents are as follows ：

0.0.0.0

1.1.1.1

2.2.2.2

The output is in the following form ：

0.0.0.0:80,1.1.1.1:80,2.2.2.2:80

[[email protected] shell]#sed -i 'N;N;s/\n/:80,/g;s/$/:80/' test.txt
[[email protected] shell]# cat test.txt
0.0.0.0:80,1.1.1.1:80,2.2.2.2


=============
#xargs： Turn multiple lines into a single line . The previous standard output is divided into parameters one by one through the pipeline , Use spaces to separate .
[[email protected] shell]#cat test.txt.bak|xargs
0.0.0.0 1.1.1.1 2.2.2.2
[[email protected] shell]#cat test.txt.bak|xargs|awk '{print $1":80,"$2":80,",$3":80"}'
0.0.0.0:80,1.1.1.1:80, 2.2.2.2:80

7.sed In command & Usage of

& Indicates the matching pattern in the replace command ： Represents the content that the pattern matches , Then you can quote 
[[email protected] shell]# echo "i have a fat cat"
i have a fat cat
[[email protected] shell]# echo "i have a fat cat"|sed 's/.at/".at"/g'
i have a ".at" ".at"
[[email protected] shell]# echo "i have a fat cat"|sed 's/.at/"&"/g'
i have a "fat" "cat"



### Add 0, Four digit numbers don't add , Pay attention to the use of word boundaries 
[[email protected] shell]# cat renxiaojing.txt
........
tanxue
zhang shuaishuai 123
zhang  456pengpeng pengpeng pengpeng pengpeng  xuyalan
jingjing jingjing jingjing
[[email protected] shell]# sed -i -r 's/\<[0-9]{3}\>/&0/g' renxiaojing.txt
[[email protected] shell]# cat renxiaojing.txt
.....
tanxue
zhang shuaishuai 1230
zhang4560pengpeng pengpeng pengpeng pengpeng  xuyalan
jingjing jingjing jingjing

# Add 2022
[[email protected] shell]# sed -n -r 's/[A-Z]/&2022/gp' test1.txt
abcH2022H2022H2022H2022dddde
abF2022K2022D2022dddde
isO2022O2022O2022ishabxil
H2022E2022P2022R2022E2022doodcm

8.\  \: Escape character

[[email protected] shell]# sed -n '/13\/Jul\/2022:16:53:58/,/13\/Jul\/2022:16:56:21/p' access.log
sed -n
[[email protected] shell]# sed -n '#13/Jul\/2022:16:53:58#,#13/Jul/2022:16:56:21#p' access.log

localhost shell]# sed -n 's#Jul#Aug#gp' access.log|head -5
 Modification time ：date -s " Time of revision "

9. label

#### What matches inside the parentheses is a label , When replacing later, you can quote , In the first parenthesis \1, And so on .
[[email protected] shell]# echo aaa bbb ccc |sed -r 's/([a-z]+) ([a-z]+) ([a-z]+)/\3 \2 \1/'
ccc bbb aaa
[[email protected] shell]# echo aaa bbb ccc |sed -r 's/([a-z]+) ([a-z]+) ([a-z]+)/\3/'
ccc
[[email protected] shell]# echo aaa bbb ccc |awk '{print $3,$2,$1}'
ccc bbb aaa

10. Output the following text in reverse order

0.0.0.0

1.1.1.1

2.2.2.2

[[email protected] shell]# tac test.txt.bak
2.2.2.2
1.1.1.1
0.0.0.0
 Method 2 ：
[[email protected] shell]# cat test.txt.bak|xargs|awk '{print $3"\n"$2"\n"$1}'
2.2.2.2
1.1.1.1
0.0.0.0

Two .grep command

1.grep There are two commonly used formats ：

The first one is ：

cat /etc/password | grep "root"

The second kind ：

grep "root" /etc/passwd

grep Commands can also be superimposed

# take /etc/passwd It didn't appear in the root and nobody Line out of
grep -v root /etc/passwd | grep -v nobody

2. frequently-used grep Options ：

-i    ignore    # Case insensitive

-v    invert    # Take the opposite , Print only what doesn't match , And the matching one doesn't print ( Only negate the whole line )

-n    number    # Show matching lines and line numbers

-c    count    # Number of output matching lines

-o    only-matching
# Show only the part of the matching line that matches the regular expression
# Often with wc -l Command with

-r    recursive    # Recursively check all files in the directory ( Include subdirectories ) Conduct grep lookup

-E    Extend    # Turn on extended regular expressions
egrep = grep -E

-A n    after    # Display the line of the matched string and the following n That's ok

-B n    before    # Display the line of the matched string and its front n That's ok
-C n    context    # Display the line of the matched string and its front and back n That's ok

#  give an example 
#  take /etc/passwd In file root Start with nobody Take out the first line 
[[email protected] /]# cat /etc/passwd | grep -E "^root|^nobody"
root:x:0:0:root:/root:/bin/bash
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin
[[email protected] /]# cat /etc/passwd | egrep "^root|^nobody"
root:x:0:0:root:/root:/bin/bash
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin

3. Matching character

[]        # A single character in the specified range

[0-9]    # matching 0 To 9 Any number in
[0-Z]    # Match all letters and numbers
[a-z]    # matching a~z Any of all lowercase letters of
[A-Z]    # matching A~Z Any one of all the capital letters of
[a-Z]    # Match any one of all uppercase and lowercase letters
[abc]    # Match a character , This character must be abc One of them

[^]        # A single character outside the specified range , Take the opposite

[^123]    # Match a character , This character is in addition to 1、2、3 All characters except

.        # Represents a single arbitrary character

.*        # Represents any arbitrary character

4. Number of matches

?        # The preceding character appears 0 Time or 1 Time
+        # The preceding character appears 1 More than once
*        # The preceding character appears any number of times
? 、* 、+ By default, only the first character can be decorated

{n}        # Indicates that the preceding character appears n Time
{n,m}    # Indicates that the preceding characters appear at least n Time , Most appear m Time
{n, }    # Indicates that the preceding characters appear at least n Time

5. Position anchoring

^        # The beginning of the anchor line
'^suda' Match all to suda Beginning line

$        # The end of the anchoring line
'suda$' Match all to suda The line at the end

^$        # Match a blank line

\b        # Word lock
\b or \<    # Anchor the beginning of a word
\b or \>    # Anchor the end of a word

# give an example
'\bsuda\b' Only match suda
'\bsuda' Only match words with suda start
'suda\b' Only match words with suda ending

6. Grouping and referencing

()        # Multiple character grouping can be realized by using parentheses
        # Use... In parentheses "|" Realize the function of or
    
# give an example
(aa|bb){2}