| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit I made several changes:
- Use structure instead of simple bson.M(interface{}). bson.M has
some shortcomings: 1) It makes the database in chaos and hard to
read, but this's not important; 2) Some entrys may has more or
less content than others, which makes it hard to decode and filt.
So I design new data structure to encode and decode. Hopes that
there's no bugs.
- Fix the way to calculate file path. The original method is to add
all the PATH entries together, that's totally wrong! PATH entry
has several types, as it shows in "objtype". I can't find it in
the kernel src code, so what i know is just "PARENT" means the dir
the file is in, while the filename itself has the path, so we
whould ignore all "PARENT"s. When the src code is found, we should
check it again.
- Fix bugs in updating. The update function of mongodb is set to
required to has a '$' such as 'set'/'push', so when we update a
whole doc, we should use replace but not update function. And,
we should never ignore the error infomation it gives us.
Hope that there's no more bugs for this Big Change.
Now its' time to write filter as well as viewer. Best wishes with
NO BUGS!
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As previous envisioned, the lost is caused by slow consumption. So
I make several changes:
- Delete all the mutexs, especially those in the mongodb. There seems
to have no necessity to use thread mutex, because execve, fork
and exit has no conflicton(really?)
- Insert all pid info into db, just ws what we do to file infos. So
we should filter out useful info from them, but this does works
to decrease lost infos.
Besides, the problem that recvfrom is always blocked may got solved.
When the machine is just started, it's blocked; but after i run the
C program to connect to ketlink connector and listen to it, both C
program and godo can recv infos well.
Also, left questions:
- Now i use many coroutine in 2nd and 3rd coroutines in the hope that
there will be less time to deal info and hear the kernel again as
quick as possible. But does it work? As we know, too much thread
will slower the program, because too frequent switch between threads
or processes.
- Sometimes the eventTable has bugs, when eoe is recvd, the pointer
in it is null. It may be out of thread confliction? But it's
unreasonable, there's only one place to delete the event, that's
just in eoe, after it's sent. Or the eoe info is got more than once?
- For some processes, when i look into /proc to find cwd and cmeline,
it has exited. If i go back to use audit for pid info, it will be hard
to distinguish between thread and process. Anyway, It doesn't matter
now, but what if?
Next step: Figure out the root fs of a docker, and its name.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit I make several changes, reasons are as follows:
- Expand netlink connector buffer size. Originally it uses only one
page of mem, but sometimes it causes "no enough buffer" errno, or
the socket is blocked long time. Now it's 20 pages.
- All file infos are thrown into database. As the last commit co-
mment, There's 2 tables, "fds" and "files". When a file discriptor
is closed, the info in fds will be found, delete, and put into the
"file" table with its close time.
Left questions:
- The netlink connector is always found blocked without any reasons.
Fix it, or replace the golang-coded connector with C program? The
key is why it's blocked. Maybe it's in the kernel src code.
- sometimes audit still losts info(not too much). For instance, I
use vim in the docker to change hello.c, the hello.c may be opened
but no close info recvd. Or, the swap file of vim, such as
.hello.c.swp or .hello.c.swx is not closed. What's more, the hello.c
is never written, but swap files are. May vim write to swap files,
and replace the origin file? Let's check it.
- Besides, when a pid exits, we should check its file discriptors
and close them all.
|
| | |
|
| |
|
|
|
|
| |
this commit i successfully catch open/close syscall, and insert them
as an independent collection in mongodb otherwise along with pids.
and now I've record those open flag "O_TRUNC" as written.
|
|
|
The Most important work during this time is to find out solution
to the out-of-order bug. Discribe it here in detail: info from
audit may be out of order, which means fork may comes after execve,
even after exit. What an absurd penomenon to see a process not yet
created to work or exit!
To deal with this problem, I've tried several ways:
- in the 2nd coroutine, when EOE msg comes, if it's a fork/clone
event, send it immediately, otherwise wait for some time(such as
100 ms). But after all it delays longer, and has other problems.
- the 2nd coroutine doesn't send directly, but record all the finished
event id in a slice, and another thread checks once every one second,
if there are sth in slice, send corresponding events in the order of
event id. But: event that happens first doesn't always has lower id
or time, for example, 1 forks 2, then 2 execve, the audit in kernel
it self may gets execve before fork(maybe fork makes other settings),
which means execve has earlier timestamp and lower event id. The out-
of-order problem is not completely resolved. If we then add delays
to non-clone event, a more serious problem happens: we must use mutex
to lock the slice recording finished event id to prevent crush between
send thread and wait thread, but the wait thread can't get the mutex
again, because there are to much clone event and frequent send!
- So I use no delay but mongodb, when an execve comes, if pid is not
recorded, just insert it and wait for the fork. It does works, but
some other works is still left to do:
- what should i do if 2 forks 3 comes before 1 forks 2? Now I
suggest it doesn't happen, but what if?
- when execve comes before fork, i recorded it, but if this process
has a parent i don't care, delete, or stays there?
Also, as mentioned above, I've add EXECVE field in process into db,
records all the execve(time, and args) from the same process. Besides,
exit_timestamp and exit_code can be caught now, but too many process
has no exit info. This is also to be fixed.
Now, let's listen to the file changed by process. Don't forget the
to-do works listed above!
|
We-unite