Skip to content

Commit dc34b70

Browse files
meecashmkozlowski
authored andcommitted
Extensions requied to run memcr as non-root user, and secure the UNIX domain socket
based communication: * Add -g -G options to set group ID for created UNIX domain sockets files. * Modify the code to limit the UNIX domain sockets file ownership and permissions in according to the provided -g -G options. * Create doc subdirectory and security_considerations.md describing the approach to secure/limit UNIX domain sockets access, and provide requried priviledges to memcr daemon when run as non-root user.
1 parent 806b72c commit dc34b70

File tree

11 files changed

+297
-29
lines changed

11 files changed

+297
-29
lines changed

README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
memcr was written as a PoC to demonstrate that it is possible to temporarily reduce RSS of a target process without killing it. This is achieved by freezing the process, checkpointing its memory to a file and restoring it later when needed.
1010

11-
The idea is based on concepts seen in ptrace-parasite and early CRIU versions. The key difference is that the target process is kept alive and memcr manipulates its memory with `madvise()` `MADV_DONTNEED` syscall to reduce RSS. VM mappings are not changed.
11+
The idea is based on concepts seen in ptrace-parasite and early [CRIU](https://github.com/checkpoint-restore/criu) versions. The key difference is that the target process is kept alive and memcr manipulates its memory with `madvise()` `MADV_DONTNEED` syscall to reduce RSS. VM mappings are not changed.
1212

1313
#### building
1414

@@ -23,7 +23,7 @@ You can enable support for compression and checksumming of memory dump file:
2323
There is also `ENCRYPT` option for building `libencrypt.so` that provides sample implementation of encryption layer based on libcrypto API. memcr is not linked with libencrypt.so, but it can be preloaded with `LD_PRELOAD`.
2424
- `ENCRYPT=1` - requires libcrypto and openssl headers
2525

26-
Ubuntu 22.04:
26+
##### compilation on Ubuntu 22.04:
2727
```
2828
sudo apt-get install liblz4-dev liblz4-1
2929
sudo apt-get install libssl-dev libssl3
@@ -40,7 +40,7 @@ make CROSS_COMPILE=arm-linux-gnueabihf-
4040
make CROSS_COMPILE=aarch64-linux-gnu-
4141
```
4242
##### yocto
43-
There is a generic `memcr.bb` file provided that you can copy into your yocto layer and build memcr as any other packet with bitbake.
43+
There is a generic `memcr.bb` recipe provided that you can copy into your yocto layer and build memcr as any other packet with bitbake.
4444
```
4545
bitbake memcr
4646
```
@@ -59,11 +59,14 @@ options:
5959
-d --dir dir where memory dump is stored (defaults to /tmp)
6060
-S --parasite-socket-dir dir where socket to communicate with parasite is created
6161
(abstract socket will be used if no path specified)
62+
-G --parasite-socket-gid group ID for parasite UNIX domain socket file, valid only for if --parasite-socket-dir provided,
63+
note: the group ID provided need to be common for: the user running memcr daemon and the user running suspended process
6264
-N --parasite-socket-netns use network namespace of parasite when connecting to socket
6365
(useful if parasite is running in a container with netns)
6466
-l --listen work as a service waiting for requests on a socket
6567
-l PORT: TCP port number to listen for requests on
6668
-l PATH: filesystem path for UNIX domain socket file (will be created)
69+
-g --listen-gid group ID for listen UNIX domain socket file, valid only in service mode for UNIX domain socket
6770
-n --no-wait no wait for key press
6871
-m --proc-mem get pages from /proc/pid/mem
6972
-f --rss-file include file mapped memory
@@ -85,3 +88,4 @@ memcr client:
8588
memcr-client -l 9000 -p 1234567 --checkpoint
8689
memcr-client -l 9000 -p 1234567 --restore
8790
```
91+
Due to high priviledges of the memcr daemon it is recommended to run memcr daemon process as non-root user with elevated Linux capabilities and permissions, the details are described in: [doc/security_considerations.md](doc/security_considerations.md)

arch/arm/linux-abi.h

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,6 @@ static long syscall3(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
6666
return r0;
6767
}
6868

69-
#if 0
7069
static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3)
7170
{
7271
register long r7 asm("r7") = nr;
@@ -80,9 +79,7 @@ static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
8079
: "memory");
8180
return r0;
8281
}
83-
#endif
8482

85-
#if 0
8683
static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4)
8784
{
8885
register long r7 asm("r7") = nr;
@@ -97,7 +94,6 @@ static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
9794
: "memory");
9895
return r0;
9996
}
100-
#endif
10197

10298
#if 0
10399
static long syscall6(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5)

arch/arm64/linux-abi.h

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,6 @@ static long syscall3(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
6666
return x0;
6767
}
6868

69-
#if 0
7069
static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3)
7170
{
7271
register long x8 asm("x8") = nr;
@@ -80,9 +79,7 @@ static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
8079
: "memory");
8180
return x0;
8281
}
83-
#endif
8482

85-
#if 0
8683
static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4)
8784
{
8885
register long x8 asm("x8") = nr;
@@ -97,7 +94,6 @@ static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
9794
: "memory");
9895
return x0;
9996
}
100-
#endif
10197

10298
#if 0
10399
static long syscall6(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5)

arch/riscv64/linux-abi.h

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,6 @@ static long syscall3(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
6666
return a0;
6767
}
6868

69-
#if 0
7069
static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3)
7170
{
7271
register long a7 asm("a7") = nr;
@@ -80,9 +79,7 @@ static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
8079
: "memory");
8180
return a0;
8281
}
83-
#endif
8482

85-
#if 0
8683
static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4)
8784
{
8885
register long a7 asm("a7") = nr;
@@ -97,7 +94,6 @@ static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
9794
: "memory");
9895
return a0;
9996
}
100-
#endif
10197

10298
#if 0
10399
static long syscall6(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5)

arch/syscall.c

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
#include <sys/types.h>
2020
#include <sys/socket.h>
2121
#include <sys/syscall.h>
22+
#include <linux/fcntl.h> /* for O_* and AT_* */
2223

2324
#if defined(__x86_64__)
2425
#include "x86_64/linux-abi.h"
@@ -88,3 +89,35 @@ long sys_gettid(void)
8889
{
8990
return syscall0(__NR_gettid);
9091
}
92+
93+
int sys_fchmod(int fd, mode_t mode)
94+
{
95+
return syscall2(__NR_fchmod, fd, mode);
96+
}
97+
98+
int sys_chmod(char* path, mode_t mode)
99+
{
100+
#ifdef __NR_fchmodat
101+
return syscall4(__NR_fchmodat, AT_FDCWD, (unsigned long)path, mode, 0);
102+
#elif defined(__NR_chmod)
103+
return syscall2(__NR_chmod, (unsigned long)path, mode);
104+
#else
105+
return -ENOSYS;
106+
#endif
107+
}
108+
109+
int sys_chown(char* path, uid_t owner, gid_t group)
110+
{
111+
#ifdef __NR_fchownat
112+
return syscall5(__NR_fchownat, AT_FDCWD, (unsigned long)path, owner, group, 0);
113+
#elif defined(__NR_chown)
114+
return syscall3(__NR_chown, (unsigned long)path, owner, group);
115+
#else
116+
return -ENOSYS;
117+
#endif
118+
}
119+
120+
int sys_getuid(void)
121+
{
122+
return syscall0(__NR_getuid);
123+
}

arch/syscall.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,9 @@ int sys_bind(int fd, struct sockaddr *addr, socklen_t len);
3030
int sys_listen(int fd, int n);
3131
int sys_exit(int error_code);
3232
long sys_gettid(void);
33+
int sys_fchmod(int fd, mode_t mode);
34+
int sys_chmod(char* path, mode_t mode);
35+
int sys_chown(char* path, uid_t owner, gid_t group);
36+
int sys_getuid(void);
3337

3438
#endif

arch/x86_64/linux-abi.h

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ static long syscall3(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
5959
return ret;
6060
}
6161

62-
#if 0
6362
static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3)
6463
{
6564
register unsigned long r10 asm("r10") = r10;
@@ -72,9 +71,7 @@ static long syscall4(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
7271
: "memory");
7372
return ret;
7473
}
75-
#endif
7674

77-
#if 0
7875
static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4)
7976
{
8077
register unsigned long r10 asm("r10") = r10;
@@ -89,7 +86,6 @@ static long syscall5(int nr, unsigned long arg0, unsigned long arg1, unsigned lo
8986
: "memory");
9087
return ret;
9188
}
92-
#endif
9389

9490
#if 0
9591
static long syscall6(int nr, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5)

doc/security_considerations.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Security considerations
2+
3+
## socket communication
4+
5+
### parasite socket
6+
7+
The memcr uses a UNIX domain socket for communication between the parasite (code injected to the suspended process) and memcr utility/daemon, let's call it parasite_socket.
8+
The parasaite_socket is a UNIX domain socket, and depends on the memcr options (`-S --parasite-socket-dir`) can be a named UNIX domain socket created in the pointed directory, or abstract UNIX domain socket.
9+
10+
Using abstract UNIX domain socket is more straightforward as do not require any option to memcr, or take care of the socket ownership and permissions, but it is less secure, as:
11+
- the name/ID of the created socket is generated as: ***memcr\<pid of the suspended process\>*** so it is easy to guess,
12+
- there is no user/permissions-based access control to abstract UNIX domain sockets, any user in the system can list them and connect to them if know the socket name.
13+
It is recommended to not use abstract UNIX domain socket in a systems where security measures should be applied.
14+
15+
Access to UNIX domain socket file can be controlled by socket file node permissions and ownership.
16+
By default parasaite_socket is created as owned by the user runnig the suspended process, with RW rights only for the owner. It is enough if memcr is run as root (so it can access any file).
17+
If the system is configured to be more secure `-G --parasite-socket-gid` option may be specified for memcr to provide group ID which will own the parasite_socket with RW access to it. This is useful for the possible solutions where memcr is run as a non-root user with elevated Linux capabilities.
18+
19+
### restore socket
20+
21+
The restore socket is a UNIX domain socket used internally by memcr to communicate between main process and forked process watching the suspended process (one instance created per suspended process).
22+
Analogically to the parasite_socket it is created as named UNIX domain socket (named ***memcrRestore\<pid of the suspended process\>***), or abstract UNIX domain socket depends on `-S --parasite-socket-dir` option.
23+
The owner of the named UNIX domain socket is memcr effective user, permissions are set to RW for owner only.
24+
25+
### daemon socket
26+
27+
The second socket is created when memcr is run as a daemon (`-l --listen option`) and is used to send the commands to memcr daemon by memcr-client utility, let's call it daemon_socket.
28+
The daemon_socket can be a UNIX domain socket created as a file node pointed by `-l` option, or TCP socket listening on port number defined with `-l` option.
29+
30+
For TCP socket, the access to the provided port can be controlled by a network access control mechanism (iptables).
31+
32+
For UNIX domain socket: it is named socket node, by default owned by effective UID and gid of the user running memcr, and having access permissions based on the umask set for the memcr process. (Note, that in most cases it means that running memcr daemon as root will require running memcr-client as root as well.)
33+
Setting the chosen group ID with `-g --listen-gid` memcr option, the file group ownership is changed to the provided gid, and file node permission is set to RW for the owner and the group.
34+
This way one can limit access to memcr daemon to the user(s) being part of the selected group. It is recommended to create a separate group for that purpose to strictly limit the access.
35+
36+
### examples
37+
38+
1. memcr daemon running as root with TCP daemon socket (port 9000), abstract UNIX domain sockets used for parasite and restore sockets:
39+
40+
```
41+
sudo memcr -zc -l 9000
42+
```
43+
44+
memcr client, run as non-root user, connects to TCP socket:
45+
46+
```
47+
memcr-client -l 9000 -p <pid> --checkpoint
48+
memcr-client -l 9000 -p <pid> --restore
49+
```
50+
51+
> [!NOTE]
52+
> no memcr daemon access control, no memcr internal sockets protection.
53+
54+
2. memcr daemon running as root with UNIX domain daemon socket, abstract UNIX domain sockets used for parasite and restore sockets:
55+
56+
```
57+
sudo memcr -zc -l /tmp/memcr/memcr.sock
58+
59+
/tmp/memcr$ ls -l
60+
total 0
61+
srwxr-xr-x 1 root root 0 gru 31 19:35 memcr.sock
62+
```
63+
64+
memcr client, run as root (to be able to connect to daemon), connects to UNIX domain socket:
65+
66+
```
67+
sudo memcr-client -l /tmp/memcr/memcr.sock -p <pid> --checkpoint
68+
sudo memcr-client -l /tmp/memcr/memcr.sock -p <pid> --restore
69+
```
70+
71+
> [!NOTE]
72+
> memcr daemon access control by /tmp/memcr/memcr.sock owner/permissions, no memcr internal sockets protection.
73+
74+
3. memcr daemon running as root with with UNIX domain daemon socket, UNIX domain sockets used for parasite and restore sockets, gid 1000 set for daemon and parasite sockets:
75+
76+
```
77+
sudo memcr -zc -l /tmp/memcr/memcr.sock -g 1000 -S /tmp/memcr -G 1000
78+
79+
/tmp/memcr$ ls -l
80+
total 0
81+
srw-rw---- 1 root user 0 gru 31 19:38 memcr.sock
82+
```
83+
84+
memcr client, run as non-root user, connects to UNIX domain socket (suspended process pid: 33239)
85+
86+
```
87+
memcr-client -l /tmp/memcr/memcr.sock -p 33239 --checkpoint
88+
89+
/tmp/memcr$ ls -l
90+
total 0
91+
srw-rw---- 1 user user 0 gru 31 19:40 memcr33239
92+
srw------- 1 root root 0 gru 31 19:40 memcrRestore33239
93+
srw-rw---- 1 root user 0 gru 31 19:39 memcr.sock
94+
95+
memcr-client -l /tmp/memcr/memcr.sock -p 33239 --restore
96+
```
97+
98+
> [!NOTE]
99+
> memcr daemon access control by /tmp/memcr/memcr.sock owner/permissions, memcr parasite and restore sockets (memcr33239, memcrRestore33239) access control by sockets file node owner/permissions
100+
101+
## Linux capabilities/filesystem permissions required by memcr to operate
102+
103+
In order to run memcr as a non-root user it is required to grant to memcr process/user Linux capabilities and filesystem nodes permissions required for memcr operation. Information provided in this section should allow to run memcr (as daemon as well) as non-root, and make your system more secure. Another step recommended for even better security is to run memcr as a daemon in a sandbox, for example using switch root or Linux Container (LXC).
104+
105+
> [!CAUTION]
106+
> Linux capabilities required to effectively freeze the process and dump its memory are real security threats - granting them to a non-root user running the memcr daemon process should be done carefully with a full understanding of the required changes and their consequences. It is recommended to create a separate user and group for memcr daemon, and selectively grant access to the /proc data of the suspended process by a dedicated group.
107+
108+
### CAP_SYS_PTRACE
109+
110+
memcr process does require Linux capability CAP_SYS_PTRACE to be able to call ptrace() in order to attach and control suspeneded process (see ptrace(2) and capabilities(7) for more infromation).
111+
112+
setcap command line utility can be used to set a memcr executable file capabilities attribute to the specified capability. This way the capability is granted by the OS to the process created by running such a file. (see setcap(8) / getcap(8) for more details).
113+
114+
```
115+
$ sudo setcap 'cap_sys_ptrace=ep' ./memcr
116+
117+
$ getcap ./memcr
118+
memcr cap_sys_ptrace=ep
119+
```
120+
121+
### /proc access
122+
123+
memcr does require access to data in /proc
124+
125+
1. Read access to /proc/kpageflags node.
126+
127+
In most systems, by default it is allowed only: for root to read /proc/kpageflags.
128+
The recommended solution here would be to add read (only) access for a dedicated group, and add only the user running memcr to this group.
129+
130+
2. Read/Write access to data in /proc/\<suspended process pid\>/:
131+
132+
* /proc/\<suspended process pid\>/maps
133+
* /proc/\<suspended process pid\>/mem
134+
* /proc/\<suspended process pid\>/ns/net
135+
* /proc/\<suspended process pid\>/pagemap
136+
* /proc/\<suspended process pid\>/status
137+
* /proc/\<suspended process pid\>/task
138+
139+
This access is granted to the user running the particular process and its default group. The quickest solution would be to add the user running the memcr to the suspended process default group. Sometimes such a solution could be too "wide", for example when this group grants access to some other resources owned by this process, which are not suppoused to be available for memcr daemon. The generation of dedicated groups for this purpose could be a better solution. A careful analysis of each case is recommended.
140+
141+
> [!NOTE]
142+
> To quickly test the memcr daemon working as non-root user memcr can be run as the same user as a process which will be suspended. In such a case it will be enough to grant the capability to the memcr file and access to /proc/kpageflags for the user used in the test.
143+

0 commit comments

Comments
 (0)