MattockFS Computer Forensics File-System

This project is maintained by pibara

What is MattockFS?

MattockFS is a Computer Forensics File-System. More accurately, it's a user-space file-system that aims to provide a major part of the underpinning of an envisioned message-passing based Computer Forensics Framework for use in a Computer Forensics laboratory. MattockFS aims to be a component in a Computer Forensics Framework aimed at medium to large scale investigations that is suitable for both professional investigation settings yet also suitable as basis for a framework that takes into account the need for compatibility with a more academic setting. MattockFS is not a full Digital Forensics Framework. Rather, MattockFS is a central component for a three part Computer Forensics Framework. It performs multiple key functionalities that a Computer Forensics Framework requires, and fulfills a linking function between the other two sub-frameworks.

MattockFS in context.

Before we describe MattockFS itself, we shall briefly visit the base components of the envisioned Computer Forensics Framework that MattockFS aims to be part of. We envision a distributed open-source Computer Forensics Framework that we shall refer to as The Mattock Computer Forensics Framework, or just Mattock for short. This framework is loosely modeled after the Open Computer Forensics Architecture (OCFA), an old Computer Forensic Framework build and open-sourced by the Dutch National Police that as a project has been orphaned many years ago. One of the core qualities of OCFA was the fact that it allowed for the integration of forensic and non-forensic data processing actors into dynamic data-processing tool-chains. Mattock aims to do the same, but using a more modern system design and applying current day data processing technology and computer forensic insights as well as over a decade worth of lessons-learned from the use of OCFA in investigations. We shall not go into full detail with respect to Mattock. All we will discus here is the top-level design of Mattock. More specifically, the way Mattock is defined to be the combination of three main sub-frameworks. The first thing we need to realize is that Mattock (like OCFA before it) aims to be a Computer Forensics Framework that allows a tool-chain for any particular piece of data to be dynamically determined based on content and meta-data. the second thing to realize is that Mattock (like OCFA before it) takes the process as worker approach. A worker is a single operating system process that acts as a worker for processing a particular type of data in a particular way. An installation may run a single instance of a particular actor, or it may run multiple instances, either on the same system or on a cluster of Mattock server node's. Unlike OCFA, Mattock takes data locality very seriously. That is, data prefers to stick to being processed on a single node and tool-chains containing a node switch are aimed to be scares. So now that we have a base idea of how Mattock aims to work for the Computer Forensics Process, we shall look at our three sub-frameworks:

It is important to note that at time of writing, MattockFS was in early aplha and neither MattockLib nor MattockNet had even one line of code written for it. I hope to make Mattock as a whole a community effort, so if you feel inclined to start a new open-source computer forensics project, MattockLib or MattockNet might be something to consider.

MattockFS will provide the per-node access to one of the data repositories, more on this below. While read-access is provided globally, MattockFS provides strict context and access control for adding and unforgeable provenance logged access to any writing operations. Combined with these data-input controls, MattockFS implements the message-bus functionality that in OCFA was handled by the Anycast Relay. It does so in the form of an in-file-system local system message bus for handing over data chunks to the next module actor in the tool-chain. MattockFS also provides both MattockLib and MattockNet with usefull page-cache and anycast load statistics that these systems may use for throttling and/or load-balancing purposes.

MattockLib is the part of the framework that allows actors to communicate with one or multiple MattockFS instances. It allows workers to register with MattockFS as an instance of a particular actor. To accept a job from the MattockFS per-actor any-cast bus, to store derived (meta-)data in a suitable repository, and to determine the next hop in the tool-chain based on distributed router functionality within MattockLib that allows MattockLib to determine the next hop based on tool-chain router state and newly acquired meta-data from the module actor. Mattocklib is also responsible properly serializing module actor meta-data for applying libmagic logic to any indeterminate content type. MattockLib is also responsible for working with the page-cache, anycast and CPU load stats from the underlying system in order to adjust its interaction with MattockFS and for throttling new-data input if appropriate. The most important thing that MattockLib provides is a standardized API for data processing actor. It is strongly suggested to anyone wanting to implement a MattockLib implementation to use the old OCFA tree-graph API as basis for an asynchronous lambda based API design.

MattockNet provides a mesh-up network between the different MattockFS instances on the network. It provides a system-load based system for networked input of digital forensic input data, and provides monitoring of system load for all the servers in the mesh-up. If there is a big difference in system load and if there are specifically suitable jobs in the anycast buses of the overloaded node, than MattockNet shall take action to migrate specific jobs to a different node.

The underlying archive.

Currently the underlying archive for MattockFS is a single large sparse file under /var/mattock/archive/0.dd . This is suitable for single-node experimental setups, but for future multi-node setups that work with MattockNet, at least two alternatives need to be implemented;

The final version of MattockFS should support both these mode's. It is suggested also that other options such as erasure-encoded distributed storage should be explored. The use of alternative (forensic) storage formats should also be considered an important topic of future study.

CarvPath annotations

If there is one joining property of all of the MattockFS features, it must be the use of CarvPath annotations. MattockFS's predecessor CarvFS introduced the concept of CarvPath annotations. A CarvPath annotation is a simple string token, suitable for use as pseudo-file-name, that encodes a list of one or more sparse or real fragments within a larger entity. In MattockFS, under the $mp/carvpath directory, Carvpath annotations are used to designate pseudo-files within the archive. Before we discuss how these are designated in MattockFS, we shall discuss the basics of CarvPath annotations.

So how are these CarvPath designations used in MattockFS? The directory structure is quite straight forward and allows for nested CarvPath usage. The directory $MP/carvpath is an un-listable directory. Two valid entities can be designated within $MP/carvpath. A valid single level CarvPath with a file extension (any extension will do), designates a usable pseudo file. Without a file-extension, a sub-directory is designated that also is unlistable. Within this sub-directory, any valid sub-entity designating CarvPath, with or without file extension, will designate a symbolic link that points to a flattened version of the designated two level CarvPath. For example '0+4096_S8192_4096+4096/2048+4096.dat' would become a symbolic link to '../2048+2048_S8192_4096+2048.dat'. This setup allows for arbitrary long nesting constructs as each nesting level is immediately flattened by the symbolic link.

Long paths

The use of long paths and their shortened representation poses a problem that may not be directly obvious. As the long path itself can often not be communicated between the Mattock workers and the file-system, and as it can not be derived back from just the hash, we need a place of shared storage that is shared between the workers and all relevant instances of the MattockFS file-system. The current implementation of MattockFS uses Redis as key/value store and expects the Mattock workers to do the same.

The tool-chain and overlapping hot CarvPath annotations.

The second most important concept within MattockFS is the concept of the tool-chain. We consider the processing of a single CarvPath annotated chunk of archive to follow the path of a dynamic tool-chain. As long as the tool-chain for any given CarvPath has not been fully completed, we consider the CarvPath to be hot. After the last actor in the toolchain completes, the CarvPath becomes cold, and MattockFS takes appropriate measures (calling possix_fadvise) to notify the kernel of this fact. Given that CarvPath annotations will often overlap (Think e-mail within mbox within ISO image within file-system partition within disk-image), a CarvPath may not actually become cold (or at least not completely become cold) when the last actor is done with it. A parent or child CarvPath may still be hot. To work with this, MattockFS maintains a reference counting stack of hot toolchain CarvPath annotations that is tightly bound to the Anycast concept within MattockFS. If the reference count for a chunk of archive goes down to zero, it is marked as cold, if it goes from zero to one it is marked as hot. This process is logged for profiling purposes into /var/mattock/log/0.refcount .

The job

in MattockFS, a tool-chain CarvPath will exist in an incarnation as a Job. A Job may be active (currently being processed by an actor), or may be part of one of the Anycast job-sets. During these incarnations, the tool-chain will build up a provenance log that will become available once the tool-chain has completed.

file-system as API and privilege separation tool.

In the later days of OCFA, CarvFS was used for data-storage. The file-system however was fully read-only and ran under the exact same UID as the modules did and writing to the raw archive was done by the kickstarting module rather than the file-system. In MattockFS we take a different approach. MattockFS runs under its own private UID and tha raw archive is owned by this UID. Writing to the archive is done through the file-system under strict and independently logged conditions. We consider the use of a seperate UID and the use of the file-system as writing and logging deputy to be essential privilege separations constructs for the overall robustness of the Mattock framework. We take the conservative approach that a single unaudited third-party module having the power to corrupt the archive or the provenance log as a result of hostile-data processing should be considered an unacceptable risk for large-scale multi-module processing setups. The data corruption capacity of the archive data by a vulnerable tool should be limited to those parts of the archive ascribable to the vulnerable tool. We thus consider privilege separation and other access control measures that help to guard this desired framework property to be of the utmost importance.

Sealed/frozen mutable data

The first data-integrity measure that MattockFS implements is the concept of sealed or frozen data. MattockFS allows each worker to allocate mutable archive storage, and only within the context of the tool-chain it is working on. The mutable data gets represented as a fixed-size writable file within MattockFS. A note to implementors: The file may be writable, it's not truncatable. After the worker is finished creating the file, MattockFS allows the worker to freeze the mutable data and obtain a CarvPath to the frozen read-only version of the file. Just as a sub-CarvPath, this CarvPath can than be entered into MattockFS as first job in a new child-entity tool-chain, remaining immutable for the remaining actors in the chain.

Provenance log

While MattockFS in general is meta-data achnostic (MattockLib generated meta-data is treated as just an other mime-type), our privileged separation needs require lab-side provenance related meta-data to be handled by MattockFS. We do this by creating a first provenance record when the first job in a tool-chain is created, appending a short provenance record for each consecutive job, and writing all records (as single-line JSON) to a provenance log file. Currently the provenance log is written to /var/mattock/log/0.provenance . In a final version of MattockFS it may proof useful to also route provenance-meta entities to a data-store module actor. There currently are some design considerations hindering an implementation of this feature though.

The Anycast message-bus

In OCFA the Anycast relay was based on a persistent priority queue implementation. In MattockFS the current Anycast implementation lacks both persistence and a working journaling system capable of restoring known good state. Implementation of a journal based restore should be considered an essential missing feature. In MattockFS we have abandoned the idea of using a priority queue. when analyzing old OCFA profiling logs, the priority queuing turned out not to yield the results desired with respect to page-cache hit-rates. Instead of infinitely grow-able queues, MattockFS provides MattockLib with hooks for throttling input instead. MattockLib can query MattockFS with respect to Anycast-set sizes and the amount of hot job data in the system. On the other end, MattockFS allows MattockLib to ask for a next job according to one or more job-select policies. Further study for optimum select-pollicies is needed and MattockFS tries to implement a few potentially useful policies in order to facilitate this research:

Not all of these policies are currently implemented and the future may show many of these policies to be a poor choice. There are multiple interdependent issues with choosing a policy and only full-scale testing with a functional Mattock framework will reveal the most optimum policy choices. The ultimate goal is to choose the policies that enable effective load balancing between nodes, limit the average page-cache miss rates, maximize the risk of spurious throttling and optimize the opportunity for beneficial opportunistic hashing. As some of these goals could end up mutually exclusive, real-data testing will become essential.

The use of sparse capabilities

In computer-programming, the use of object references is common good. A reference, other than a raw pointer, can not be made up by tricks like guessing and pointer-arithmetic. A reference as such is unguessable, at least in commonly used memory safe languages. In a file-system most file names are guessable names. Given our privilege separation needs, we find ourselves in a situation where we desire a safe reference between a Mattock Module and program state maintained in MattockFS. We don't want a vulnerable corruptible worker to be in the position where it could grab hold of and corrupt state belonging to other worker. In order to avoid that, we use the concept of sparse capabilities for our MattockLib/MattockFS interaction. As shown in MinorFS, just using sparse capabilities is insufficient a guard against such issues. MattockFS implements a sparse capability based interface as FS based API, but the use of mandatory access control (like AppArmor) should be considered to fully secure the forensic process from integrity attacks through vulnerable actors.

Opportunistic hashing

Given the extensive use of partially overlapping hot CarvPath entities within MattockFs, and given the fact that each high-level read/write translates to a low-level read/write, we have the opportunity to try and make hashing an opportunistic process. Whenever a chunk of low-level data is accessed, its access may be transposed to ghost-IO actions on each of the hot entities that overlap the accessed section. Under specific conditions, these ghost-IO actions may be used to further an opportunistic hashing process for each of those entities. If at any point in time the opportunistic hashing reaches the full size of the entity, than opportunistic-hashing will have been fully successful. Further, at any point in time, the opportunistic hashing index of a hot entity may be queried, so that only the last part of a file would need to be read if it becomes needed by a worker or routing action. All completed opportunistic hashing actions are available during the life-time of the tool-chain. Additionally, successful hashing is logged to /var/mattock/log/0.ohash

The file-system based API

So now that we have all the theory out of the way, lets show how this all comes together as a file-system. We define our API largely in the form of sparse capabilities, special '.ctl' pseudo files and extended attributes. Think of ctl files as objects, sparse capabilities as object references and extended attributes as methods. If you grasp these three methods, understanding the FS-level API of MattockFS, and translating it to a programming-language specific API should become relatively trivial. In some rare cases the extended attribute interface does not fully map to a method paradigm. Communicating through extended attributes is limited to getter and setter functionality. To functionality that can be mapped as void functions with parameters and non-void functions without. This means that, perforce, a non-void function with parameters needs to be expressed as two separate operations. Further when we want to communicate more than one fields, we opt for a simple semicolon separated composite string to do so. So let's look at the directory structure of our file-system first:

We already discussed the whole carvpath/ directory structure in the section on CarvPath annotations, so lets focus on the FS-based API part.

carvpath/$CARVPATH.$EXT : This read-only file contains all the data under the designated CarvPath.

mattockfs.ctl : This control file gives access to two pieces of global full-file-system data. It has the following extended attributes:

actor/$ACTORNAME.ctl : These per-modulename control files serve as a double point of access. One for registering a process as worker and a second one for pre-submit information that may be relevant for throttling purposes.

actor/$ACTORNAME.inf : These per-modulename control files serve as attenuated versions of the above ctl files. They are meant to be used for throttling info by workers other than the one the actor name refers to. This so that a least authority MAC protected setup can allow modules access to the trottling info without granting powerfull priviledges to other modules.

worker/$WORKERSPARSECAP.ctl : This control is meant to be the primary control object for a registered worker. We have the following attributes:

job/$JOBSPARSECAP.ctl : This temporarily valid control represents a Job that the worker is currently in the process of processing.

mutable/$MUTABLEDATACAP : This temporary file is a mutable fixed-size entity that is writable but can not be truncated. The file may be written to sparsely during its lifetime.

Helping out / Support / Contact

As stated, MattockFS aims to become part of the envisioned community-effort project Mattock. A community page was made on Google+. If you wish to contribute, or if you desire any information or support related to Mattock or MattockFS, please consider joining this G+ community.