path: root/docs/sym_upload_v2_protocol.md

# Introduction

The `sym_upload` tool is able to operate in `sym-upload-v2` protocol mode, in
addition to the legacy protocol (which will be referred to as `sym-upload-v1`
for the rest of this document). For now `sym-upload-v2` is HTTP/REST-based but
it could be extended to operate over gRPC instead, in the future.

# Table of Contents
* [Why](#why)
* [How](#how)
   * [Uploading](#uploading)
     * [Uploading with `sym_upload`](#uploading-with-sym_upload)
     * [Uploading with curl](#uploading-with-curl)
     * [Serving the `sym-upload-v2` protocol](#serving-the-sym-upload-v2-protocol)
       * [Authenticate using `key`](#authenticate-using-key)
       * [Symbol `checkStatus`](#symbol-checkstatus)
       * [Upload `create`](#upload-create)
       * [Uploading the symbol file](#uploading-the-symbol-file)
       * [Upload complete](#upload-complete)


# Why

Using `sym_upload` in `sym-upload-v2` protocol mode has the following features
beyond `sym-upload-v1`:
  * Authentication via `key` (arbitrary secret).
  * Symbol identifier (product of `debug_file` and `debug_id`, as recorded in
output from `dump_syms`) can be checked against existing symbol information on
server. If it's present, then the upload is skipped entirely.

# How

## Uploading

### Uploading with `sym_upload`

Uploading in `sym-upload-v2` protocol mode is easy. Invoke `sym_upload` like
```
$ ./sym_upload -p sym-upload-v2 [-k <API-key>] <symbol-file> <API-URL>
```

Where `symbol-file` is a symbol file created by `dump_syms`, `API-URL` is the
URL of your `sym-upload-v2` API service (see next section for details), and
`API-key` is a secret known to your uploader and server.

For more options see `sym_upload --help`.

### Uploading with curl

As an example, if:
  * Your API's URL was "https://sym-upload-api".
  * Your service has assigned you `key` "myfancysecret123".
  * You wanted to upload the symbol file at "path/to/file_name", with
`debug_file` being "file_name" and `debug_id` being
"123123123123123123123123123". Normally you would read these values from
"path/to/file_name", which in turn was generated by `dump_syms`.

Then you might run:
```
$ curl https://sym-upload-api/symbols/file_name/123123123123123123123123123:checkStatus?key=myfancysecret123
```

And, upon seeing that this `debug_file`/`debug_id` combo is missing from symbol
storage then you could run:
```
$ curl --request POST https://sym-upload-api/uploads:create?key=myfancysecret123
```

Which returns `upload_url` "https://upload-server/42?creds=shhhhh" and
`upload_key` "42". Next you upload the file directly like:
```
$ curl -T path/to/file_name "https://upload-server/42?creds=shhhhh"
```

Once the HTTP PUT is complete, run:
```
$ curl --header "Content-Type: application/json" \
    --request POST \
    --data '{symbol_id:{"debugFile":"file_name",'\
        '"debugId":"123123123123123123123123123"}}' \
    https://sym-upload-api/uploads/42:complete?key=myfancysecret123
```

### Serving the `sym-upload-v2` Protocol

The protocol is currently defined only in HTTP/REST. There are three necessary
REST operations to implement in your service:
* `/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>`
* `/uploads:create?key=<key>`
* `/uploads/<upload_key>:complete?key=<key>`

#### Authenticate Using `key`

The query string arg `key` contains some secret that both the uploader and
server understand. It is up to the service implementer to decide on what
constitutes a valid `key`, how the uploader acquires one, and how to handle
requests made with invalid ones.

#### Symbol `checkStatus`

```
/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>
```

This operation expects an empty (or no) JSON payload in the request.

This operation should return the status of the symbol file uniquely identified
by the given `debug_file` and `debug_id`. JSON schema:
```
{
    "type": object",
    "properties": {
        "status": {
            "type": "string",
            "enum": ["STATUS_UNSPECIFIED", "MISING", "FOUND"],
            "required": true
        }
    }
}
```

Where `MISSING` denotes that the symbol file does not exist on the server and
`FOUND` denotes that the symbol file exists on the server.

#### Upload `create`

```
/uploads:create?key=<key>
```

This operation expects an empty (or no) JSON payload in the request.

This operation should return a URL that uploader can HTTP PUT their symbol file
to, along with an "upload key" that can be used to notify the service once the
file upload is completed. JSON schema:
```
{
    "type": "object",
    "properties": {
        "upload_url": {
            "type: "string",
            "required": true
        },
        "upload_key": {
            "type": "string",
            "required": true
        }
    }
}
```

Since this REST API operation can be authenticated via the `key` query string
arg, the service can return a URL that encodes permission delegation to the
upload endpoint resource and thereby constrain the ability to upload to those
with valid `key`s.

#### Uploading the Symbol File

Note that the actual symbol upload step is _not_ part of the REST API. The
upload URL obtained in the above operation is meant to be used as the endpoint
for a normal HTTP PUT request for the contents of the symbol file. Once that
HTTP PUT request is completed use the upload `complete` operation.

#### Upload `complete`

```
/uploads/<upload_key>:complete?key=<key>
```

This operation expects a JSON payload in the HTTP request body with the
following schema:
```
{
    "type": "object",
    "properties": {
        "symbol_id": {
            "type": "object",
            "properties": {
                "debug_file": {
                    "type": "string",
                    "required": true
                },
                "debug_id": {
                    "type": "string",
                    "required": true
                }
            }
        }
    }
}
```

This operation should cause the symbol storage back-end (however implemented)
to consume the symbol file identified by `upload_key`. It is up to the service
implementation to decide how uploads are assigned `upload_key`s and how to
retrieve a completed upload by its `upload_key`. If the symbol file cannot be
found, is malformed, or the operation cannot be completed for any other reason
then an HTTP error will be returned. JSON schema of non-error responses:
```
{
    "type": "object",
    "properties": {
        "result": {
            "type": string,
            "enum": ["RESULT_UNSPECIFIED", "OK", "DUPLICATE_DATA"],
            "required": true
        }
    }
}
```

Where `OK` denotes that the symbol storage was updated with the new symbol file
and `DUPLICATE_DATA` denotes that the symbol file data was identical to data
already in symbol storage and therefore nothing changed.