Lfs | S3 Account

  • Configure concurrent transfers:

  • Example AWS CLI upload (for manual object upload):

  • Verbose debug for LFS:

  • Generate pre-signed URL in Python (boto3):

    import boto3
    s3 = boto3.client('s3')
    url = s3.generate_presigned_url('put_object',
                                    Params='Bucket': 'my-lfs-bucket', 'Key': key,
                                    ExpiresIn=3600)
    
  • Ensure metadata and pointer OIDs remain consistent.
  • Features you might want to generate or configure for an LFS S3 account include: lfs s3 account

    JSON policy example (restrict to a single bucket and prefix):

    
      "Version": "2012-10-17",
      "Statement": [
    "Sid": "LFSObjectAccess",
          "Effect": "Allow",
          "Action": [
            "s3:PutObject",
            "s3:GetObject",
            "s3:DeleteObject"
          ],
          "Resource": "arn:aws:s3:::my-lfs-bucket/lfs/objects/*"
        ,
    "Sid": "ListBucketIfNeeded",
          "Effect": "Allow",
          "Action": "s3:ListBucket",
          "Resource": "arn:aws:s3:::my-lfs-bucket",
          "Condition": 
            "StringLike": 
              "s3:prefix": "lfs/objects/*"
    ]
    

    Inside the LFS chroot (or final system): Configure concurrent transfers:

    # Install awscli from source or binary
    wget https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip
    unzip awscli-exe-linux-x86_64.zip
    ./aws/install
    

    An “S3 account” usually means an AWS account with S3 enabled, or an IAM user within an AWS account.

    Assuming 5 GB source bucket, 20 GB binary cache, 2 GB logs, 10 GB images: Example AWS CLI upload (for manual object upload):

    Total < $2/month for a hobbyist LFS user.