Add optional encryption (#67)

Add more illustrations about using the local disk as the backend (#62)

* add docker-compose.local.yml

add an example yml file for using local storage as a backend

Signed-off-by: Hongyu Li <543306408@qq.com>

* add the illustration of using local disk as backend

Signed-off-by: Hongyu Li <543306408@qq.com>

---------

Signed-off-by: Hongyu Li <543306408@qq.com>

Reflect auth headers back into response objects (#59)

If an Authentication header is present in the request to the server, then reflect it in the response object from the server. If this is not present the git-lfs client will attempt to send unauthenticated requests to the server. With this change rudolfs can be protected behind a Kubernetes NGINX Ingress resource that uses the nginx.ingress.kubernetes.io/auth-url annotation.

Refactor local smoke test (#47)

I'm not 100% sure it is logically safe to just destroy the .git/lfs directory as was done in the prior version of this test.

I think the same test logic can be accomplished in a safer and more plausible way by simply making another clean repo.

Fixup logic for "faulty"

The faulty test is interesting but it needs refactor before it will work with current tokio.

This patch attempts that refactor.

I didn't notice that the "faulty" test was failing to build before due to the feature flag being off.

Update environment variables in docker-compose.minio.yml

* Minio changed their environment variable names so reflect that
* Need to pass in AWS_REGION env var for this to work with minio

Bump to humansize 2

Trivial refactor required.

Bump backoff to version 0.4

This upgrade required a very minor refactor due to updated type structure.

Bump hex to 0.4

This upgrade required a minor refactor but the code does seem cleaner to me now.

Bump rusoto to 0.48

Need to remove `Sync` from ByteStream bounds to fix compile error.

Run cargo update

Bump all dependencies to best current version of spec.

Fix clippy warnings

Also turns all future warnings into errors so these don't creep in
again.

Migrate to github actions (#37)

Support multipart S3 uploads (#36)

Adds support for >5GB uploads. Each part is also retried if there is a transient S3 error.

Co-authored-by: Jason White <github@jasonwhite.io>

Support X-Forwarded-Host header

Fixes #31. This should allow Rudolfs to work with load balancers.

Unbreak the S3 backend

There was a check missing for `upload_url` causing all uploads to bypass
the encryption process. Then, when the same object gets downloaded and
decrypted, the SHA256 verification fails causing an infinite download
loop.

This also adds a test to help catch issues like this in the future.

Allow Rudolfs to be used as a library

This makes integration testing easier.

Use S3 signed URL for upload & allow specifying download domain (#27)

Update askama and rand crates (#30)

Update backoff crate (#29)

Add support for using AWS credentials when running under EKS (#26)

Update docker-compose.yml (#24)

Update all dependencies (#22)

Add support for local storage (#21)

Make --cache-dir optional (#20)

If not specified, then no cache will be used at all.

Use async-await syntax (#18)

Fix custom region logic (#14)

Previously, this code used Rusoto to get the default region as an object then, if there was a custom endpoint turn that region into a Region::Custom with the previously acquired default region name as the custom name. However, Region::default() performs internal checks to filter out names that do not match the official AWS region names and defaults to 'us-east-1' if it's not in the list. This result in custom region names always being ignored and defaulting to 'us-east-1'. The fix changes the logic around this to first check if a custom endpoint is used and then check for custom region names - if no custom endpoint is specified, it simply uses the AWS default.

Add retry on initial S3 connection

This helps alleviate race conditions on startup of multiple services.

Update rusoto dependency

This release changed the way errors work. It also fixes chunk
duplication for streams.

Add support for custom S3 endpoints with $AWS_S3_ENDPOINT (#6)

This allows the usage of minio and ceph as S3 backends.

Fix cache ending up with incomplete objects

Also adds fault injection to help track down rare bugs related to errors
in backend services.

Fixes #5.