1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
|
```
_ ___ _
| | |__ \ | |
_ __ ___ __ _| |_ ) | ___ __ _____| |__ Trashing your meta,
| '_ ` _ \ / _` | __| / / |___|\ \ /\ / / _ \ '_ \ keeping your data,
| | | | | | (_| | |_ / /_ \ V V / __/ |_) | within your browser.
|_| |_| |_|\__,_|\__|____| \_/\_/ \___|_.__/
```
This is an online version of [mat2](https://0xacab.org/jvoisin/mat2).
Keep in mind that this is a beta version, don't rely on it for anything
serious, yet.
# Demo instance
There is a demo instance deployed a [mat2-web.dustri.org](https://mat2-web.dustri.org).
Please don't upload any sensitive files to it.
# Vue Frontend
There is a SPA Frontend available at https://0xacab.org/jfriedli/mat2-quasar-frontend. It consumes
the RESTful API of this project. As a fallback for non JS users it redirects to this web app.
To set it up checkout the [Readme](https://0xacab.org/jfriedli/mat2-quasar-frontend/blob/master/README.md).
# How to deploy it?
mat2 is available in [Debian stable](https://packages.debian.org/stable/mat2).
```
# apt install uwsgi uwsgi-plugin-python3 git mat2
# apt install nginx-light # if you prefer nginx
# apt install apache2 libapache2-mod-proxy-uwsgi # if you prefer Apache2
# cd /var/www/
# git clone https://0xacab.org/jvoisin/mat2-web.git
# mkdir ./mat2-web/uploads/
# chown -R www-data:www-data ./mat2-web
```
Since [uWSGI](https://uwsgi-docs.readthedocs.io/en/latest/) isn't fun to
configure, feel free to copy
[this file](https://0xacab.org/jvoisin/mat2-web/tree/master/config/uwsgi.config)
to `/etc/uwsgi/apps-enabled/mat2-web.ini` and
[this one](https://0xacab.org/jvoisin/mat2-web/tree/master/config/nginx.config)
to `/etc/nginx/sites-enabled/mat2-web`.
Nginx is the recommended web engine, but you can also use Apache if you prefer,
by copying [this file](https://0xacab.org/jvoisin/mat2-web/tree/master/config/apache2.config)
to your `/etc/apache2/sites-enabled/mat2-web` file.
Configure the following environment variables:
- `MAT2_ALLOW_ORIGIN_WHITELIST=https://myhost1.org https://myhost2.org`
Note that you can add multiple hosts from which you want to accept API requests. These need to be separated by
a space. **IMPORTANT:** The default value if the variable is not set is: `Access-Control-Allow-Origin: *`
- `MAT2_MAX_FILES_BULK_DOWNLOAD=10` Max number of files that can be grouped for a bulk download.
Note: Each file has a max file size of 16mb
- `MAT2_MAX_FILE_AGE_FOR_REMOVAL=900` Seconds a file in the upload folder is kept.
After that it will be deleted. Default `15 * 60`
- `MAT2_WEB_DOWNLOAD_FOLDER` Define the upload folder path. Defaults to: `./uploads/`
Finally, restart uWSGI and your web server:
```
systemctl restart uwsgi
systemctl restart nginx/apache/…
```
It should now be working.
Note for reverse proxies: Include the Host header to ensure all generated urls are correct.
e.g. for Nginx: `proxy_set_header Host $host;`
# Deploy via Ansible
If you happen to be using [Ansible](https://www.ansible.com/), there's an
Ansible role to deploy mat2-web on Debian, thanks to the amazing
[systemli](https://www.systemli.org/en/index.html) people:
[ansible-role-mat2-web](https://github.com/systemli/ansible-role-mat2-web)
The role installs mat2-web as a uWSGI service, and runs it as a dedicated
system user, installs bubblewrap to sandbox mat2 and creates a garbage
collector cronjob to remove leftover files. Besides, it can create a
[dm-crypt](https://en.wikipedia.org/wiki/Dm-crypt) volume with random key for
the uploads folder, to ensure that the uploaded files won't be recoverable
between reboots.
# Development
Install docker and docker-compose and then run `docker-compose up` to setup
the docker dev environment. Mat2-web is now accessible on your host machine at `localhost:5000`.
Every code change triggers a restart of the app.
If you want to add/remove dependencies you have to rebuild the container.
# RESTful API
## Upload Endpoint
**Endpoint:** `/api/upload`
**HTTP Verbs:** POST
**Body:**
```json
{
"file_name": "my-filename.jpg",
"file": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
}
```
The `file_name` parameter takes the file name.
The `file` parameter is the base64 encoded file which will be cleaned.
**Example Response:**
```json
{
"inactive_after_sec": 120,
"output_filename": "fancy.cleaned.jpg",
"mime": "image/jpg",
"key": "81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161",
"secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
"meta": {
"BitDepth": 8,
"ColorType": "RGB with Alpha",
"Compression": "Deflate/Inflate",
"Filter": "Adaptive",
"Interlace": "Noninterlaced"
},
"meta_after": {},
"download_link": "http://localhost:5000/download/81a541f9ebc0233d419d25ed39908b16f82be26a783f32d56c381559e84e6161/44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c/fancy.cleaned.jpg"
}
```
## Supported Extensions Endpoint
**Endpoint:** `/api/extension`
**HTTP Verbs:** GET
**Example Response (shortened):**
```json
[
".asc",
".avi",
".bat",
".bmp",
".brf",
".c",
".css",
".docx",
".epub"
]
```
**Endpoint:** `/api/download/bulk`
This endpoint allows you to bulk download several files
which you uploaded beforehand. Note that the `download_list`
MUST contain more than two files. The max length is configurable
(default is 10).
**HTTP Verbs:** POST
**Body:**
```json
{
"download_list": [
{
"file_name": "uploaded_file_name.jpg",
"key": "uploaded_file_key",
"secret": "uploaded_file_secret"
}
]
}
```
The `file_name` parameter takes the file name from a previously uploaded file.
The `key` parameter is the key from a previously uploaded file.
**Example Response:**
```json
{
"inactive_after_sec": 120,
"output_filename": "files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip",
"mime": "application/zip",
"key": "5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4",
"secret": "44deb60b5febbd466e042f4172d36bcc5f7eb2eb6791d6e93191c378a381ae7c",
"meta_after": {},
"download_link": "http://localhost/api/download/5ee4cf8821226340d3d5ed16bd2e1b435234a9ad218f282b489a85d116e7a4c4/files.2cd225d5-2d75-44a2-9f26-e120a87e4279.cleaned.zip"
}
```
# Docker
There are two Dockerfiles present in this repository. The file called `Dockerfile.development` is used for development
and `Dockerfile.production` is used for production deployments.
You can find the automated docker builds in the registry of this
repository: https://0xacab.org/jvoisin/mat2-web/container_registry
### Building the production image
Build command: `docker build -f Dockerfile.production -t mat-web .`
Run it: `docker run -ti -p8181:8080 --read-only --tmpfs /tmp --tmpfs /run/uwsgi --tmpfs=/app/upload --security-opt=no-new-privileges --security-opt=seccomp=./config/seccomp.json mat-web:latest`
This does mount the upload folder as tmpfs and servers the app on `localhost:8181`.
##### Podman
Build: `podman build -f Dockerfile.production -t matweb-podman .`
Run: `podman run -ti -p8181:8080 --read-only --tmpfs /tmp --tmpfs /run/uwsgi --tmpfs=/app/upload --security-opt=no-new-privileges --security-opt=seccomp=./config/seccomp.json matweb-podman:latest`
# Configuration
The default settings from `main.py` may be overridden by adding a `config.py`
file and add custom values for the relevant flask config variables. E.g.:
```
MAX_CONTENT_LENGTH = 32 * 1024 * 1024 # 32MB
```
See [Flask configuration docs](http://exploreflask.com/en/latest/configuration.html)
for further information.
# Custom templates
You can override the default templates from `templates/` by putting replacements
into the directory path that's configured in `app.config['CUSTOM_TEMPLATES_DIR']`
(default `custom_templates/`).
# Threat model
- An attacker in possession of the very same file that a user wants to clean,
along with its names, can perform a denial of service by continually
requesting this file, and getting it before the user.
- An attacker in possession of only the name of a file that a user wants to
clean can't perform a denial of service attack, since the path to download
the cleaned file is not only dependent of the name, but also the content.
- The server should do its very best to delete files as soon as possible.
# Licenses
- mat2-web is under MIT
- The [raleway](https://github.com/impallari/Raleway/) font is under OFL1.1
- [normalize.css](https://github.com/necolas/normalize.css/) is under MIT
- [skeleton](http://getskeleton.com/) is under MIT
|
