S3Drive - Importing rclone crypt configs not working

Dear S3Drive team I recently installed the latest version d the app and noticed that when i export and import my rclone configs to my new device, the crypt Volume no longer displays my existing encrypted files and folders. Is there something I need to consider when exporting/importing the configs? I tried importing the configs to round sync and it was working fine

nourshobier

Dear S3Drive team I recently installed the latest version d the app and noticed that when i export and import my rclone configs to my new device, the crypt Volume no longer displays my existing encrypted files and folders. Is there something I need to consider when exporting/importing the configs? I tried importing the configs to round sync and it was working fine

Thanks for your bug report. When you import configs into new device, are your endpoints appearing in the Accounts tab? Can you try toggling off/on the Mount in the Settings?

Tom

Thanks for your bug report. When you import configs into new device, are your endpoints appearing in the Accounts tab? Can you try toggling off/on the Mount in the Settings?

Hi Tom, yes they appear in the accounts tab, i tried toggling mount as well but unfortunately with no luck

Hi again, Thanks for your feedback. We're already investigating the issue, however just to clarify things.

the crypt Volume no longer displays my existing encrypted files and folders.

Does it display anything, is it empty or it doesn't appear on the mount list at all? Do you have any other mounts appearing correctly? Are they encrypted/unencrypted? What's your Android version? (edited)

I have both encrypted and unencrypted for gdrive, the unencrypted one works fine. But the encrypted one doesn't show old files, however, new encrypted files are displayed fine with no issue My android version is 16 on pixel phone

8:06 PM

The encrypted one is displayed in the mount list as well as the unencrypted one

Do those new files also show up in RoundSync? It sounds like the keys used to encrypt/decrypt the files weren't successfully imported in S3Drive, maybe it didn't parse some special character correctly or something. So essentially it's now using different keys. With these keys it can't decrypt your existing files, and files created using these keys can't be decrypted when using the original keys. So in RoundSync, where you said importing the configs worked as expected, you would see your existing files, but those new files wouldn't show up.

I exported the rclone configs from s3drive on my pc and imported them to my phone. That said, i tested out ur theory and u are right, i don't see the files created from s3drive. Do u maybe know how to resolve the situation on my phone? Btw, the s3drive on pc works fine with no issue

Thanks for the hint @Unlawful Cactus What’s your backend that crypt points to? Having said that, we’ll double check if password gets properly serialised/deserialised during import export. (edited)

This is what my config file endpoints looklike [gdrive] type = drive ... [gcrypt] type = crypt remote = gdrive:encrypted-folder ...

nourshobier

This is what my config file endpoints looklike [gdrive] type = drive ... [gcrypt] type = crypt remote = gdrive:encrypted-folder ...

But the encrypted one doesn't show old files, however, new encrypted files are displayed fine with no issue

a) Are pre-existing encrypted files displayed correctly in the S3Drive UI (not the Android mount)? Does it work on both devices? b) Are pre-existing encrypted files displayed correctly in the origin device (the one from which you exported config) in the Android mount? c) If your files were encrypted with S3Drive to start with. Can you double check if your gcrypt also contain: filename_encoding = base64? It is setting used by S3Drive and crucial for the filepath/name decryption. Bear in mind that if filepath decryption fails (either due to incorrect password or/and settings) these files will appear with mangled name in the S3Drive UI listing, but won't appear anywhere were Rclone is involved (mounts, sync etc.). The other important fact is that if you have existing unencrypted files and then enable encryption, similar side-effect will kick in, that is your unencrypted files will no longer be visible in the mount/sync, but will appear in the S3Drive listing. You can verify if file is encrypted by using Properties (check attached screenshot). d) Can you take your password and use rclone reveal command to double check if it's the one that you've set it to and that it matches the E.g. for: password = Aoyfgxb_7wZbm-kQ_PoOupZrqjd50A line, you type: rclone reveal Aoyfgxb_7wZbm-kQ_PoOupZrqjd50A (you need https://rclone.org/install/ installed on one of your machines). I hope we can get to this eventually. (edited)

Install

Rclone Installation

nourshobier

I exported the rclone configs from s3drive on my pc and imported them to my phone. That said, i tested out ur theory and u are right, i don't see the files created from s3drive. Do u maybe know how to resolve the situation on my phone? Btw, the s3drive on pc works fine with no issue

Since you asked: You basically want to make sure the config of your crypt looks exactly the same on your phone as it does on your PC (especially the passwords). You can view the config from the Accounts screen > context menu of the crypt > Show .ini config. If they look different, and [on your phone] reinstalling S3Drive and re-importing your config didn't solve the issue, I would try adding the crypt through Accounts > + > Custom provider and paste the correct config there. Of course, with everything you do, be careful not to permanently lose any data. I don't think you need rclone reveal, as identical obfuscated passwords from the config file will reveal identical passwords, so checking if the configs match exactly should be good enough and save you the hassle of installing rclone. To be clear, I'm not affiliated with S3Drive, just a fellow user. I'm going to shut up now because I'm just interfering with Tom's approach to solve this and that might just annoy him and confuse you

Good luck!

Unlawful Cactus

Since you asked: You basically want to make sure the config of your crypt looks exactly the same on your phone as it does on your PC (especially the passwords). You can view the config from the Accounts screen > context menu of the crypt > Show .ini config. If they look different, and [on your phone] reinstalling S3Drive and re-importing your config didn't solve the issue, I would try adding the crypt through Accounts > + > Custom provider and paste the correct config there. Of course, with everything you do, be careful not to permanently lose any data. I don't think you need rclone reveal, as identical obfuscated passwords from the config file will reveal identical passwords, so checking if the configs match exactly should be good enough and save you the hassle of installing rclone. To be clear, I'm not affiliated with S3Drive, just a fellow user. I'm going to shut up now because I'm just interfering with Tom's approach to solve this and that might just annoy him and confuse you

Good luck!

I very much agree with what @Unlawful Cactus said. Using: Show .ini config on both ends shall reveal any discrepancies, whereas using Custom provider is an alternative way to import .ini where config can be pasted in instead of requiring to create a file first. Regarding:

I don't think you need rclone reveal, as identical obfuscated passwords from the config file will reveal identical passwords

At the moment only reveal will show the exact password, and two seemingly different obscured passwords might point to the same password. Usually during Rclone import we tell it to wrap (aka obscure) password, as we've thought that some users may want the ability of to provide it as plaintext (even though it is technically wrong from the format standpoint of view). This works fine, however for already obscured passwords this would end up in "double" wrap, which doesn't work. ... so what we do, is we always try to reveal / unwrap and then wrap it again with Rclone. This allows us to import both obscured and unobscured passwords, however it always changes as wrapping nonce is changing as well. Perhaps we could find a way to preserve existing obscure version. (edited)

O, thanks for correcting me Tom, I wasn't aware the obfuscation mechanism uses a nonce “stored verbatim at the beginning of the obscured password” as I've now read in rclone's documentation. Nice to learn something new!

Tom

But the encrypted one doesn't show old files, however, new encrypted files are displayed fine with no issue

a) Are pre-existing encrypted files displayed correctly in the S3Drive UI (not the Android mount)? Does it work on both devices? b) Are pre-existing encrypted files displayed correctly in the origin device (the one from which you exported config) in the Android mount? c) If your files were encrypted with S3Drive to start with. Can you double check if your gcrypt also contain: filename_encoding = base64? It is setting used by S3Drive and crucial for the filepath/name decryption. Bear in mind that if filepath decryption fails (either due to incorrect password or/and settings) these files will appear with mangled name in the S3Drive UI listing, but won't appear anywhere were Rclone is involved (mounts, sync etc.). The other important fact is that if you have existing unencrypted files and then enable encryption, similar side-effect will kick in, that is your unencrypted files will no longer be visible in the mount/sync, but will appear in the S3Drive listing. You can verify if file is encrypted by using Properties (check attached screenshot). d) Can you take your password and use rclone reveal command to double check if it's the one that you've set it to and that it matches the E.g. for: password = Aoyfgxb_7wZbm-kQ_PoOupZrqjd50A line, you type: rclone reveal Aoyfgxb_7wZbm-kQ_PoOupZrqjd50A (you need https://rclone.org/install/ installed on one of your machines). I hope we can get to this eventually. (edited)

Hi @nourshobier, I thought to ask how it's going. I was wondering if these instructions were any useful for you or perhaps you got stuck somewhere? Thanks!

Hey Tom, apologies for the late reply a) Are pre-existing encrypted files displayed correctly in the S3Drive UI (not the Android mount)? Does it work on both devices?

Yes it works fine on my pc

b) Are pre-existing encrypted files displayed correctly in the origin device (the one from which you exported config) in the Android mount?

Yes they are working fine on that PC. However, when installed s3drive on a different PC and imported the exported configs, i faced the same issue on my phone till I configured everything with rclone cli, only then s3 drive started to work fine again.

c) If your files were encrypted with S3Drive to start with. Can you double check if your gcrypt also contain: filename_encoding = base64? It is setting used by S3Drive and crucial for the filepath/name decryption.

No, everything was configured before s3drive using rclone cli before

d) Can you take your password and use rclone reveal command to double check if it's the one that you've set it to and that it matches the

Yes the passwords look good as well

nourshobier

Hey Tom, apologies for the late reply a) Are pre-existing encrypted files displayed correctly in the S3Drive UI (not the Android mount)? Does it work on both devices?

Yes it works fine on my pc

b) Are pre-existing encrypted files displayed correctly in the origin device (the one from which you exported config) in the Android mount?

Yes they are working fine on that PC. However, when installed s3drive on a different PC and imported the exported configs, i faced the same issue on my phone till I configured everything with rclone cli, only then s3 drive started to work fine again.

c) If your files were encrypted with S3Drive to start with. Can you double check if your gcrypt also contain: filename_encoding = base64? It is setting used by S3Drive and crucial for the filepath/name decryption.

No, everything was configured before s3drive using rclone cli before

d) Can you take your password and use rclone reveal command to double check if it's the one that you've set it to and that it matches the

Yes the passwords look good as well

Sorry I am sort of getting lost in it. I am failing to understand if issue lies solely in import function or something to do with Android mount or perhaps some config incompatibility between S3Drive and Rclone CLI since you've said:

No, everything was configured before s3drive using rclone cli before

Can we start again? Is this issue reproducible if you set up new back-end (e.g. gdrive), then crypt, and then use export/import? In such case can you provide Minimal Reproducible Example, that is mention step by step how to reproduce the issue? Much appreciated!

Few months ago before i install s3drive, i already had my rclone cli installed and the crypt mount was configured already. Later i found s3drive and installed it on my pc first then on my phone and exported the rclone configs using cli to my phone. Now when i do the same thing my phone doesn't show the crypt mount pre-existing files anymore

Dear Tom, I have an update from my side. I tried to run rclone dump and import the configs as json to s3drive and it worked

I didn't expect it to be so simple

2

nourshobier

Dear Tom, I have an update from my side. I tried to run rclone dump and import the configs as json to s3drive and it worked

I didn't expect it to be so simple

That's great news! You're right, we support both formats, be it JSON or INI, but in principle there shouldn't be any difference in importing either. Do you think that S3Drive has some issues with importing .INI? Did you use import function for JSON or perhaps used: Custom provider button where you can paste config as text?

I'm not sure what went wrong with the ini format TBH, but i exported it from rclone and imported it to s3drive directly

Hi, please let me join your discussion as I faced the same problem on the S3Drive iOS app; that means the „crypt“ drive does no show any files; instead is shows the „It’s pretty empty“ message. Per se the rclone setup is up and running on Linux and macOS machines. Uploading the rclone.config file brings the remote and the crypt_remote to the iOS app. The remote works perfectly fine showing encrypted files whose naming is consistent with what I see under Linux and macOS. When it comes to the crypt_remote also Linux and macOS are perfectly consistent, but the iOS app sees no files. Also when I add a file to the crypt_remote via the app and check on (e.g.) Linux I there get an error message!? I both tried to setup the crypt_remote either via enabling e2e encryption and entering the password manually and by uploading the rclone config (.ini; JSON was not accepted) file used for Linux. I also exported the configuration and did a rclone reveal to check the password for consistency. It is what it is expected to be. Any idea how this could be solved?

Thanks, we still haven't exactly found and/or reproduced this issue, so we'll need probably need more info.

„It’s pretty empty“ message

If underlying remote has in fact some files then this indicated that either passphrase isn't correct or this (https://rclone.org/crypt/#crypt-filename-encoding) wasn't set to base64 You've said:

and did a rclone reveal to check the password for consistency. It is what it is expected to be.

Can you please make sure that you get the config out to compare from S3Drive using: Show .ini config function in the Accounts page?

Also when I add a file to the crypt_remote via the app and check on (e.g.) Linux I there get an error message!?

What's the error message?

(.ini; JSON was not accepted)

We did a quick test where both .ini and .json (which is produced like rclone config dump > config.json) worked fine. Did you get any error message? Did you try import by going to the Accounts page, clicking "+" (top-right corner), clicking Custom provider and pasting either .ini or .json in the text area? We're willing to solve this issue, but we're struggling to reproduce it, thanks for your help!

Hi Tom, thanks for your prompt reply. Wrt your questions I can say the following: I checked the rclone reveal also against the „Show .ini config“ output but the results also match. But what I found out is that when I use the e2e activation from the app I cannot set the proper path for the crypt_remote („remote = remote_drive:dir1/dir2“) But this works fine when I use the import rclone.config. As said I can add files to the crypt_remote from the app. Executing rclone ls crypt_remote: on the Linux machine results in the following error msg „Skipping undecryptable file name: bad PKCS#7 padding - too long“ Access to the test file (created within the app) was possible regardless of whether the base64 encoding or filename_encryption=true was specified in the .ini file. I tested all combinations. It also has no impact on the error msg on the Linux machine. Also I tried to import by going to the Accounts page and clicking on „Custom provider“. But when I paste to result form the rlcone config dump command into the text area (below the „Rclone .ini config“ header) the „add“ button was not working. It only worked when I paste a rclone config format I also tried the „Import Rclone configuration“ with the .json file which did not work either. In case I did something wrong I can’t figure out what it is…

PS: When I use the S3Drive app on macOS I don’t get an error message above (this happens only when using rclone on the command line). The test file simply is not there when using the crypt_remote. Using the remote I can see all (old ones and new test file) encrypted files.

dz3456

PS: When I use the S3Drive app on macOS I don’t get an error message above (this happens only when using rclone on the command line). The test file simply is not there when using the crypt_remote. Using the remote I can see all (old ones and new test file) encrypted files.

As said I can add files to the crypt_remote from the app. Executing rclone ls crypt_remote: on the Linux machine results in the following error msg „Skipping undecryptable file name: bad PKCS#7 padding - too long“

PS: When I use the S3Drive app on macOS I don’t get an error message above (this happens only when using rclone on the command line). The test file simply is not there when using the crypt_remote. Using the remote I can see all (old ones and new test file) encrypted files.

This is an indication that location contains both encrypted and unencrypted files. S3Drive will be able to display both, however Rclone removes them from the listing as it assumes broken (according to encryption settings), instead of unencrypted.

Also I tried to import by going to the Accounts page and clicking on „Custom provider“. But when I paste to result form the rlcone config dump command into the text area (below the „Rclone .ini config“ header) the „add“ button was not working. It only worked when I paste a rclone config format

You're right JSON doesn't work with "Custom Provider" button, but it will be improved in a next release. On the other hand the Rclone import from file should support both .json and .ini. (edited)

It does indeed and a miracle occurred… (maybe you can add a hint in your next release )

dz3456

It does indeed and a miracle occurred… (maybe you can add a hint in your next release )

Great to hear that, but I am still somewhat confused, as in principle .ini import either from file or by using Custom Provider button should render same effects as .json and I got the impression you've tried that route. (edited)

I agree and it does seem strange indeed (but as you confirmed the Custom Provider path does not like JSON; so both ways can‘t be identical)

Hi. I've had the same problem. It seems that the iOS app changes the "password" value in config during import. I've spent a few days digging into it. Also I can't mount crypt volume to Files app, other mounts work. I've purchased the lifetime subscription just to be able to access pcloud crypt volume from my iPad, hope these issues will be fixed. BTW Do you have traditional tickets or email support? It's such a bad practice to require paid customers to sign up for another social media account with a phone number just to get support

1:52 AM

I've tried to import both ways, using copy/paste into "custom config" and "import rclone config". "password" value in "show ini" section changes to completely different after import. Also, if I rename the storage, it changes once again. I hope it will help

1:54 AM

If you need more info, I can create new volumes and make screenshots or videos of how exactly it changes. Anyway I have backups (edited)

Izzy

Hi. I've had the same problem. It seems that the iOS app changes the "password" value in config during import. I've spent a few days digging into it. Also I can't mount crypt volume to Files app, other mounts work. I've purchased the lifetime subscription just to be able to access pcloud crypt volume from my iPad, hope these issues will be fixed. BTW Do you have traditional tickets or email support? It's such a bad practice to require paid customers to sign up for another social media account with a phone number just to get support

Thanks for your feedback.

I've had the same problem. It seems that the iOS app changes the "password" value in config during import.

"password" value in "show ini" section changes to completely different after import.

This is related to reobscure feature which allows user to use both plaintext and obscured password in the config. I've described how it works in a comment above: https://discord.com/channels/1069654792902815845/1402048402027577475/1403075936014045204 In principle different value in the password field still points to the same password, this can be confirmed by using rclone reveal command

Also I can't mount crypt volume to Files app, other mounts work.

Unfortunately there is a technical issue which we haven't solved just yet. In principle, rclone crypt loaded as a module doesn't fit into 20MB memory limit imposed by iOS: https://discord.com/channels/1069654792902815845/1359944785544351774/1359977486212923513 We'll either need to find some hacky way to improve memory usage and/or reimplement crypt using Swift, but that would likely work only for S3 and would be used solely for iOS Files.

I've purchased the lifetime subscription just to be able to access pcloud crypt volume from my iPad, hope these issues will be fixed.

Thank you for your purchase. As mentioned above we aim to fix that, but it's not easy/straightforward. We'll try to dedicate more time into researching possible solutions.

BTW Do you have traditional tickets or email support? It's such a bad practice to require paid customers to sign up for another social media account with a phone number just to get support

We do, it's support@s3drive.app. Feel free to choose what works best for you. We use Discord as default so we can then collaboratively work together and that way we have a growing knowledgebase about S3Drive which benefits anyone.

Also, if I rename the storage, it changes once again. I hope it will help

Do you mean that storage name doesn't persist?