My Digital Garden

Moving Mastodon Media to Object Storage

Moving Mastodon Media to Object Storage

Introduction

Setup on Digitial Ocean

  1. Determine sub-domain we will serve media from media.social.synesthesia.co.uk

  2. In Digital Ocean control panel DNS, create DNS A entry for media.social.synesthesia.co.uk to public IP of droplet

  3. In DO control panel create space, note name (synesthesia-media)

  4. In Digitial Ocean control panel API/Spaces Keys create a new key with same name as bucket, make secure note of Access Key and Secret key

  5. On server, create new file /etc/nginx/sites-availavble/media

    proxy_cache_path /tmp/nginx_mstdn_media levels=1:2 keys_zone=mastodon_media:100m max_size=1g inactive=24h;
server {
 listen 80;
 listen [::]:80;
 server_name media.social.synesthesia.co.uk;
 root /home/mastodon/live/public;
 location /.well-known/acme-challenge/ { allow all; }
 return 301 https://media.social.synesthesia.co.uk$request_uri;
 access_log /dev/null;
 error_log /dev/null;
 }
 server {
 listen 443 ssl http2;
 listen [::]:443 ssl http2;
 server_name media.social.synesthesia.co.uk;

 access_log /var/log/nginx/mstdn-media-access.log;
 error_log /var/log/nginx/mstdn-media-error.log;

 # Add your certificate and HTTPS stuff here
 ssl_protocols TLSv1.2 TLSv1.3;
 ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
 ssl_prefer_server_ciphers on;
 ssl_session_cache shared:SSL:10m;

 # Uncomment these lines once you acquire a certificate:
 # ssl_certificate     /etc/letsencrypt/live/media.YOUR_DOMAIN.com/fullchain.pem;
 # ssl_certificate_key /etc/letsencrypt/live/media.YOUR_DOMAIN.com/privkey.pem;

 location /synesthesia-media/ {

     proxy_hide_header      Strict-Transport-Security;
     proxy_cache mastodon_media;
     proxy_cache_revalidate on;
     proxy_buffering on;
     proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
     proxy_cache_background_update on;
     proxy_cache_lock on;
     proxy_cache_valid 1d;
     proxy_cache_valid 404 1h;
     proxy_ignore_headers Cache-Control;
     add_header X-Cached $upstream_cache_status;
     proxy_pass https://synesthesia-media.ams3.digitaloceanspaces.com/;
 }
}

  6. ln -s /etc/nginx/sites-available/media /etc/nginx/sites-enabled/media

  7. nginx -t

  8. systemctl restart nginx

  9. Get certificate #TODO

  10. edit /etc/nginx/sites-available/media and uncomment certificate lines

  11. su mastodon

  12. add following in to Mastodon /home/mastodon/live/.env.production

    S3_ENABLED=true
S3_BUCKET=synesthesia-media
AWS_ACCESS_KEY_ID=ENTER_YOUR_DIGITAL_OCEAN_API_ACCESS_KEY_HERE
AWS_SECRET_ACCESS_KEY=ENTER_YOUR_DIGITAL_OCEAN_API_SECRET_KEY_HERE
S3_PROTOCOL=https
S3_HOSTNAME=media.social.synesthesia.co.uk
S3_ENDPOINT=https://ams3.digitaloceanspaces.com

  13. exit (back to root)

  14. systemctl restart mastodon*

  15. su mastodon

  16. s3cmd --configure
    should look something like

    screenshot of s3cmd config

  17. Copy media content to bucket
    s3cmd sync /home/mastodon/live/public/system/ s3://YOUR_S3_BUCKET_NAME_DO_SPACES_NAME/ --verbose

  18. Set ACLs tp public
    s3cmd setacl s3://YOUR_S3_BUCKET_NAME/ --acl-public --recursive

  19. remove local copies
    rm -rf /home/mastodon/live/public/system

See also