anynines website



Benjamin Guttmann

Published at 05.08.2019

How-To’s & Tutorials

Concourse: Configure S3 Resource to Work With Alibaba Cloud OSS

Hi everyone, today I struggled with the configuration of the Concourse S3 resource to work with an Alibaba Cloud OSS bucket.

Therefore, I decided to document the issues I faced and how I solved them so maybe someone else can benefit from my pain ;-)

Let’s start!

On first view it looks very straight forward, add the configuration needed and voilà it should work.
The configuration I used looked like the following:

Code Example

2bucket: my-a9s-test-bucket
4region_name: eu-central-1

_NOTE: The endpoint is crucial if you want to configure the resource for s3 compatible (non aws) resources.

After uploading the pipeline I ended up with the following error :

Afterwards, I decided to adjust the endpoint value to ``. That resolved the original issue but afterwards I saw the following error message:

My first thought was that the URL built up with the provided configuration somehow got messed up and is checking the wrong place. To debug this issue I enabled the access logs for the OSS buckets. Now I could see that I received requests with the following URI:

‘/my-a9s-test-bucket/?marker=&prefix= ‘

So it seems like the resource attaches the bucket name to the endpoint, which might be okay for AWS S3 bucket URLs, but not for Alicloud OSS.

So one more adjustment was missing to get the resource to work, replace the bucket name with ‘/’.
Like this, the resource will check the root directory of your bucket.
So my final configuration looks like:

Code Example

2bucket: ‘/’
4region_name: eu-central-1

After I uploaded the pipeline with the updated configuration I was finally able to see the files of my OSS bucket.

I hope this quick guide helped some of you that were also struggling with the S3 resource configuration for OSS buckets.

One quick additional node, as of now it seems like versioning is not supported for s3 buckets, that means the usage of `versioned_file` property will end up in an arrow looking something like this:

© anynines GmbH 2024


Privacy Policy


© anynines GmbH 2024