111 lines
4.0 KiB
Markdown
111 lines
4.0 KiB
Markdown
# 3Dverse test task
|
|
|
|
## Solution
|
|
|
|
### Technology choosen
|
|
|
|
1. [deno](https://deno.com/) instead of nodejs as platform.
|
|
2. [deno kv](https://deno.com/kv) for database.
|
|
3. [nix](https://nixos.org/) flake as build system
|
|
|
|
|
|
### Development environment
|
|
If the developper has nix flake installed. `nix develop` will install
|
|
all the system dependencies. Otherwise, `deno` is required; hey,
|
|
and psrecord are two useful tool to benchmark the server.
|
|
|
|
### Running the tests.
|
|
|
|
The best way to build the test is to run `nix build .#test` which
|
|
will output a report in `./result`. Otherwise,
|
|
`deno test --unstable tests` should run all the tests.
|
|
|
|
|
|
### Running the project.
|
|
|
|
|
|
The main file to run the server is `src/server.ts`. To command line
|
|
`deno run --unstable --allow-net src/server.ts` will run the server locally.
|
|
One can add the flag `--v8-flags=--max-heap-size=50` to limit the
|
|
allowed memory to, in this example, 50mb.
|
|
|
|
|
|
In a similar fashion, it is possible to run the application without
|
|
checking out the code using the command line
|
|
|
|
```shell
|
|
deno run --v8-flags=--max-heap-size=50 --unstable --allow-net https://gitlab.com/didiercrunch/3dverse-testtask/-/raw/master/src/server.ts
|
|
```
|
|
|
|
## Architecture
|
|
|
|
The two main layers of abstraction are the `FileService` and `BlobService`.
|
|
The `BlobService` persist data as naivly as possible. It does so by
|
|
chuncking the data and saves it in Deno KV. The `FileService` manages
|
|
the file metadata such as the path, creation date, ...
|
|
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
title: Persist file
|
|
client->>RestAPI: HTTP request
|
|
RestAPI->>FileService: Persist file metadata
|
|
FileService->>BlobService: Persist bytes
|
|
```
|
|
|
|
The transformation part is done by the `WaveFrontService` and is 100%
|
|
agnostic of the persistace layer.
|
|
|
|
|
|
## Api
|
|
The REST api is defined in the `restapi.ts` file. The Api tries
|
|
to be as RESTful as possible.
|
|
|
|
| endpoint | method | params | description |
|
|
|:--------|:--------|:------|:----|
|
|
| `/v1/files` | `GET` | none | Lists all the files (no pagination) |
|
|
| `/v1/files/:fileId` | `GET` | none | Get the full file description |
|
|
| `/v1/files/:fileId` | `DELETE` | none | Delete the file |
|
|
| `/v1/files/` | `POST` | `path` in url params and the file content as body | Create a new file with specified path and content. |
|
|
| `/v1/files/:fileId` | `PUT` | `{path: newPath}` | Change the path of the file. |
|
|
| `/v1/content/:fileId/raw` | `GET` | none | Download the file |
|
|
| `/content/:fileId/transform` | `GET` | `scaleX`, `scaleY`, `scaleZ`, `translateX`, `translateY`, `translateZ` url params | Will download the transformed file |
|
|
|
|
|
|
## Todo
|
|
|
|
#### Data deletion
|
|
Deleting data in these types of application creates a massive race condition. Basically,
|
|
if someone is deleting a file while someone else is accessing it, the user that deletes
|
|
the file should not break the user that access the file.
|
|
|
|
The solution to implement here is a kind of garbage collector of the "blobs". This
|
|
is not trivial as it is required to add two new indexed; an index that specify if
|
|
a blob is refered by a (not deleted) file and an index that specify if a blob
|
|
is scheduled to be read.
|
|
|
|
#### HTTP Cache
|
|
Caching is very important in these type of applications. The api
|
|
is made to be easy to add proper cache.
|
|
|
|
#### Swagger
|
|
Apparently this needs to be done.. god knows how much the author of this
|
|
line hates swagger.
|
|
|
|
#### Frontend
|
|
Hooking the frontend is not yet done.
|
|
|
|
#### Implementing sqlite backend
|
|
Deno KV has been choosen by mistake. It was designed to only be used for
|
|
testing. Once the author discovered that Deno KV was a thin layer about
|
|
FoundationDB, the most perfect database, it was necessary to use it.
|
|
|
|
Nonetheless, it is important to notice that Deno KV is backed by a wasm
|
|
verison of sqlite locally and suffers from lock contention when multiple
|
|
requests read from the database.
|
|
|
|
A good ol' WAL sqlite should do the job.
|
|
|
|
#### A CLI (Or something like it)
|
|
The end user should be able to change on which port the process is runnning wihout
|
|
changing the code. A good CLI is a good fit for this. |