S3Drive
Community / general
For all on-topic discussion about S3Drive or related storage providers.
1
We're pending major release with multiple new features being added. It will also include photo backup from Android/iOS to S3. Different sync modes including two-way sync will be available ~April 2023 - https://s3drive.canny.io/feature-requests/p/syncbackup-options-two-way-one-way
1 Thanks for your help!
1
https, where as IP addr will default to http.
Filename encryption is on our Roadmap and we have a working prototype already. https://s3drive.canny.io/feature-requests/p/filenamefilepath-encryption (ETA ~April 2023).
We're making further research to understand standards or well established implementation in that area, so we can stay compatible.
The sharing functionality is based on S3 presign URLs, their limitation is that the signature can't be valid longer than 7 days, so every 7 days new link would have to be generated. We're researching how to overcome this limitation. For instance we could combine this with a link shortener, so there is single link that doesn't change, but under the hood we would regenerate the destination link as needed.
The encrypted share link has the master key at the end after the # character and looks like this:
https://s3.us-west-004.backblazeb2.com/my-s3drive/.aashare/hsnwye5bno3p/index.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=004060aad6064900000000044%2F20230214%2Fus-west-004%2Fs3%2Faws4_request&X-Amz-Date=20230214T095014Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=abdcd875e2106ee54c6a1d1851617c7e694e121464c5ca9023526ce2836be595#GKSGYX4HGNAd4nTcXb/GIA==
What it does it tries to load the encrypted asset as usual, but it's not aware per se if an asset is encrypted. In the background JavaScript tries to fetch the asset and replaces the one on the screen with decrypted version. It looks like it has failed on your side. Can you go to the console (right-click -> inspect element) to see if there is anything abnormal (that is error in the Console or different than 200 status code in any of the network requests).
1. I'm happy to hear that folder name and file name is on the near roadmap for completion. Usability problem solved
2. The S3 presign URL concept is interesting. Can a link be done for a time period shorter than a week? I assume the user would need to have their program opened and some kind of timer to check if any links need to be signed again. Then the shortened link would need to be updated. It's a good idea actually but would want to make sure it's done in a trustless or trust minimized way. Also be careful deploying resources that you have to pay for as then you'd have ongoing costs. It depends what your monetization plan is long term I guess.
3. If the link is presigned, am I right to assume that there is no ability to revoke the sharing?
4. I'm actually not familiar. What is PhotoSync?
5. The link appears to work fine in Chrome but in Firefox it stops loading the page faster. If I click refresh, the image appears as it should. Strange but seems like all is fine. The things I would typically be sharing are not photos so it's not really a priority issue for me.
6. I'd love it if S3 drive had a different favicon so I can more easily distinguish it. I'd love it more if it was mounted as a drive letter in my file explorer. 
Actually, I think what I'd personally use most is if I could right click, go to "send to" and have S3 drive as a destination. I'd have to think about if I'd want it in the main folder or prompt me where to put it. I'd probably choose to have a popup to prompt me where I want it saved within S3 drive.
1. I'm happy to hear that folder name and file name is on the near roadmap for completion. Usability problem solved
2. The S3 presign URL concept is interesting. Can a link be done for a time period shorter than a week? I assume the user would need to have their program opened and some kind of timer to check if any links need to be signed again. Then the shortened link would need to be updated. It's a good idea actually but would want to make sure it's done in a trustless or trust minimized way. Also be careful deploying resources that you have to pay for as then you'd have ongoing costs. It depends what your monetization plan is long term I guess.
3. If the link is presigned, am I right to assume that there is no ability to revoke the sharing?
4. I'm actually not familiar. What is PhotoSync?
5. The link appears to work fine in Chrome but in Firefox it stops loading the page faster. If I click refresh, the image appears as it should. Strange but seems like all is fine. The things I would typically be sharing are not photos so it's not really a priority issue for me.
6. I'd love it if S3 drive had a different favicon so I can more easily distinguish it. I'd love it more if it was mounted as a drive letter in my file explorer. Actually, I think what I'd personally use most is if I could right click, go to "send to" and have S3 drive as a destination. I'd have to think about if I'd want it in the main folder or prompt me where to put it. I'd probably choose to have a popup to prompt me where I want it saved within S3 drive. 
1
1. I'm happy to hear that folder name and file name is on the near roadmap for completion. Usability problem solved
2. The S3 presign URL concept is interesting. Can a link be done for a time period shorter than a week? I assume the user would need to have their program opened and some kind of timer to check if any links need to be signed again. Then the shortened link would need to be updated. It's a good idea actually but would want to make sure it's done in a trustless or trust minimized way. Also be careful deploying resources that you have to pay for as then you'd have ongoing costs. It depends what your monetization plan is long term I guess.
3. If the link is presigned, am I right to assume that there is no ability to revoke the sharing?
4. I'm actually not familiar. What is PhotoSync?
5. The link appears to work fine in Chrome but in Firefox it stops loading the page faster. If I click refresh, the image appears as it should. Strange but seems like all is fine. The things I would typically be sharing are not photos so it's not really a priority issue for me.
6. I'd love it if S3 drive had a different favicon so I can more easily distinguish it. I'd love it more if it was mounted as a drive letter in my file explorer. Actually, I think what I'd personally use most is if I could right click, go to "send to" and have S3 drive as a destination. I'd have to think about if I'd want it in the main folder or prompt me where to put it. I'd probably choose to have a popup to prompt me where I want it saved within S3 drive. I'm happy to be an early user and maybe able to contribute somehow along the way. I'm happy to be an early user and maybe able to contribute somehow along the way. 1 1
1
1 1 1
1 
1 2 1 1
1
1 1
1 1 1 1
1rclone for accessing my files or backing up my photos on a day to day basis.... and I am not afraid of CLIs. 
1
garage... and there is no way to provide region in S3Drive.
It seems that we may add additional form field to specify the region.toml file like this: s3_region = "us-east-1".
We auto-detect region from the endpoint URL and have a way to detect custom region from MinIO.... and if it doesn't work we use the most common default which is us-east-1.
1 1
1 1
The Discord server is hard to join from the app since it opens a webview, I'd recommend opening the invite externally so the Discord application can pick it up, or at least the standard web browser.
Trying out the application with MinIO, so far so good, pleased by the rename function that likely performs a copy + delete under the hood. It does what one would expect from an S3 client, no ads either, really liking it. .aainit file is our write test, as well as ETag response validation (which is required for not yet released syncing features), as some providers (talking mostly about iDrive E2 with SSE enabled) don't generate valid ETags. BTW. Would you like S3Drive to support read-only mode?
Regardless, we will try to improve clarity of this operation, so user feels more confident that we're not doing some shady write/reads.
Speaking of Trash itself, likely this week starting on Android first there will be a Settings option to disable Trash feature altogether (which is a soft-delete emulation, but slow and pointless if bucket already supports versioning). Versioning UI with restore options will come little bit later. .aainit file it's fine, but I'd prefer if the app saved the test results locally then deleted the file. I want to be able to write files so I wouldn't use a read-only mode, and we can always create read-only access keys if we want to be sure that's how the app will behave! I'm very interested by the share link expiry slider or date picker though, I never share for 7 days, it's either a smaller duration or permanent.
Cool, I don't mind not having the versioning UI yet, but had to delete my file versions + the trash versions to cleanup my bucket so… yeah, trash is cool but I assume most people who want that have versioning enabled. I assume you already have quite a few buckets on various providers to test your features, but I can provide a MinIO one if it could be of interest.
There was a 2nd folder with an HTML page in it, not sure what it was about but same thing I'd say, that's probably the least expected action from an S3 browser… While I audited the actions and indeed didn't find anything malicious, that could get me assassinated by my colleagues if I ever connected a more important bucket to the app. 
.aainit file it's fine, but I'd prefer if the app saved the test results locally then deleted the file. I want to be able to write files so I wouldn't use a read-only mode, and we can always create read-only access keys if we want to be sure that's how the app will behave! I'm very interested by the share link expiry slider or date picker though, I never share for 7 days, it's either a smaller duration or permanent.
Cool, I don't mind not having the versioning UI yet, but had to delete my file versions + the trash versions to cleanup my bucket so… yeah, trash is cool but I assume most people who want that have versioning enabled. I assume you already have quite a few buckets on various providers to test your features, but I can provide a MinIO one if it could be of interest.
There was a 2nd folder with an HTML page in it, not sure what it was about but same thing I'd say, that's probably the least expected action from an S3 browser… While I audited the actions and indeed didn't find anything malicious, that could get me assassinated by my colleagues if I ever connected a more important bucket to the app. s3 ls even though headObject couldn't retrieve it as a valid S3 entry. I am curious if you came across of something similar.
... but seriously, it's actually one of the coolest things I've been working on in my 10+ years career and seeing already such positive reception we've no intentions to stop. 
1 I love that one: https://xkcd.com/927/
I love that one: https://xkcd.com/927/ .aainit file it's fine, but I'd prefer if the app saved the test results locally then deleted the file. I want to be able to write files so I wouldn't use a read-only mode, and we can always create read-only access keys if we want to be sure that's how the app will behave! I'm very interested by the share link expiry slider or date picker though, I never share for 7 days, it's either a smaller duration or permanent.
Cool, I don't mind not having the versioning UI yet, but had to delete my file versions + the trash versions to cleanup my bucket so… yeah, trash is cool but I assume most people who want that have versioning enabled. I assume you already have quite a few buckets on various providers to test your features, but I can provide a MinIO one if it could be of interest.
There was a 2nd folder with an HTML page in it, not sure what it was about but same thing I'd say, that's probably the least expected action from an S3 browser… While I audited the actions and indeed didn't find anything malicious, that could get me assassinated by my colleagues if I ever connected a more important bucket to the app. 1.aainit file being nuked (delete file itself + all its versions) once the init is done and raw presigned URL sharing
The Discord server is hard to join from the app since it opens a webview, I'd recommend opening the invite externally so the Discord application can pick it up, or at least the standard web browser.
Trying out the application with MinIO, so far so good, pleased by the rename function that likely performs a copy + delete under the hood. It does what one would expect from an S3 client, no ads either, really liking it. 
Fair enough, we'll plan this in our items along with the slider which needs to be done anyway.
1headObject and get the envelope AES keys.... so it must be a toggle with some warning. It would then simply return the Blob that's stored on S3, regardless of what's inside. 
1
.aainit file it's fine, but I'd prefer if the app saved the test results locally then deleted the file. I want to be able to write files so I wouldn't use a read-only mode, and we can always create read-only access keys if we want to be sure that's how the app will behave! I'm very interested by the share link expiry slider or date picker though, I never share for 7 days, it's either a smaller duration or permanent.
Cool, I don't mind not having the versioning UI yet, but had to delete my file versions + the trash versions to cleanup my bucket so… yeah, trash is cool but I assume most people who want that have versioning enabled. I assume you already have quite a few buckets on various providers to test your features, but I can provide a MinIO one if it could be of interest.
There was a 2nd folder with an HTML page in it, not sure what it was about but same thing I'd say, that's probably the least expected action from an S3 browser… While I audited the actions and indeed didn't find anything malicious, that could get me assassinated by my colleagues if I ever connected a more important bucket to the app. .s3drive_bucket_read_test) and verify the response instead of trying to write a file.
Slider now works, so it's possible to set expiry time shorter than maximum of 7 days. There is an option to use raw preshared URLs.
We've also introduced basic Version UI. It is now possible to preview the revisions. In a next update we will allow opening, preview, deleting and restoring to particular version.
Thank you for these suggestions, they were great and helped us to validate it all !
... and as always we're open for a feedback.
1
... and we're already finding some versioning implementation differences between providers. E.g. MinIO version restore works differently than Backblaze/Wasabi. There are couple bugs (or features?) when deleting entries that we've found on Backblaze... we'll have to deal somehow with all that... but that's fine which extends versioning features, modifies behaviour of copy/rename/move/trash function to seamlessly support versioning and don't duplicate version, also there is an option to restore specific versions or delete object versions to save some space. There is also improved error handling, so in case of an Object Lock there will be an error message and relevant Log which can be viewed by long-tapping the app version number.
Versioning capabilities are configurable in the Settings.
There is a space usage indicator in the left panel / drawer.
There is a new Recent section, which displays all recently modified files.
Search function is improved, so the index is built faster and also includes implicit folder search (e.g. if there is a file: folder/file.txt, but folder/ entry doesn't explicitly exists, it is still searchable)
There is an option to hide files starting with: .
As usual there are couple other performance improvements and bugfixes.
We would love to hear how are you finding new changes and if version management during file operations is what you would expect. 
1 1Hide "." files
Show all files, including starting with the dot.
Hide files starting with the dot character
To:
Hide dotfiles
Show all files, including ones starting with a dot.
Hide files starting with the dot character. 1 1 1
1
In theory this could work. Basically some private key hidden behind the API could generate a verifiable token which would be stored on the user's bucket (and likely device for little bit of redundancy) and could be then verified. The issue that I see so far is that not every user would want their bucket to be written to with some "licensing" file. 
feature_flags int that computes to an array of pro features with bitwise operations, easy on your API and authentication gateway or whatever you do behind the scenes. 
community ! 
S3Drive version (1.2.5) on Web which uses improved rendering engine. Once it loads once all subsequent loads shall be much quicker and the UI shall be way more responsive. I would be glad to hear if you find it an improvement 1
s3.<region>.amazonaws.com 1
1
null response, which is somewhat expected. I would expect to get the SSL related error instead. 1OS Error: CERTIFICATE_VERIFY_FAILED: self signed certificate. And I indeed have self-signed certificate but I followed your instructions from https://github.com/s3drive/app/issues/19 (https://proxyman.io/posts/2020-09-29-Install-And-Trust-Self-Signed-Certificate-On-Android-11) and my browser on Andriod recognizes this certificate (if I go to minio browser, my Chrome is fine with the cert). But S3Drive continues to fail with the same error.
I'm using the latest version. 
Multiple clients (S3Drive being one of them) and multiple back-ends, all speaking 
️ the same language.
If you need my input on anything, just let me know !
It make sense to apply these improvements before focusing on background upload, so there is a stable base. 1 1 2
1
1 
1# Obscure password
echo "YourPlaintextPassword" | rclone obscure -
# Add it to Rclone config, config file location: `rclone config file`
[s3drive_remote]
type = s3
provider = Other
access_key_id = <access_key_id>
secret_access_key = <secret_access_key>
endpoint = <endpoint>
region = <region>
[s3drive_crypt]
type = crypt
filename_encoding = base64
remote = s3drive_remote:<bucket_name>
password = <obscuredPassword>
filename_encryption = standard
directory_name_encryption = true
suffix = none
Then you can use: s3drive_crypt as your remote encrypted location.
Please note that whilst we support both encrypted and unencrypted files in the same location, Rclone doesn't seem to like the mix and won't display existing unencrypted files for the encrypted remote. In such case it's better to either keep everything encrypted globally or have dedicate paths with encrypted-only or unencrypted-only files. 1
1 1
1
1
I will try out ️filename_encoding = base64
suffix = none
By default the Rclone's encoding is base32: https://github.com/rclone/rclone/blob/88c72d1f4de94a5db75e6b685efdbe525adf70b8/backend/crypt/crypt.go#L140 unless overriden by the config creator. filename_encoding = base64
suffix = none
By default the Rclone's encoding is base32: https://github.com/rclone/rclone/blob/88c72d1f4de94a5db75e6b685efdbe525adf70b8/backend/crypt/crypt.go#L140 unless overriden by the config creator. 
1filename_encoding = base64
suffix = none
By default the Rclone's encoding is base32: https://github.com/rclone/rclone/blob/88c72d1f4de94a5db75e6b685efdbe525adf70b8/backend/crypt/crypt.go#L140 unless overriden by the config creator. 
️
I upvoted${aws:username} by anything you want, be it a variable or a fixed bucket name, there unfortunately isn't any group name variable 
One policy for a group and then just add users
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::${aws:username}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::${aws:username}/*"
]
}
]
}
I will write this to my wiki and make a Post on Lemmy and Reddit. I am surely not the only one steuggeling with this
1 
Contributor role, it isn't much but still a nice way to recognize individuals who go out of their way to help the project out, what do you think about it? 
I read you need to have your profil public in your settings 