S3Drive - On Android, can't select root from external storage for Sync source

Hi, On Android, when setting up a Sync, the "Local" buttons opens a file picker that does not allow to select the root of storage or folders like Download. Instead asks to create a sub directory. Problematic to set up a full backup. And other sync solutions don't seem to have the problem. Seems linked to legacy storage access mode, or maybe using the file picker. Thanks,

Domi changed the channel name: On Android, can't select root from external storage for Sync source 2/9/2025 9:29 AM

I've added this as an item for our team to check. We've used directory picker to stay compliant (and per directory permission model) with Android SDK's, however we've recently been given broader permissions, which may possibly allow us to select root location as well. (edited)

1

Tom

I've added this as an item for our team to check. We've used directory picker to stay compliant (and per directory permission model) with Android SDK's, however we've recently been given broader permissions, which may possibly allow us to select root location as well. (edited)

Yes, it seems that apps global storage permissions were introduced in Android 11, and stabilized from 12 on. Thanks for taking into consideration. (edited)

Ah, and thinking about it : for this global / root of external storage access to be usefull, there is probably going to be something to do for the sync mechanism : with other rclone based solutions, sync from the root of external storage generates at least two access errors on Android/data and on Android/obb. Either it needs error handling or having the option to exclude some subdirectories from the sync (equivalent to --exclude='Android/data/' --exclude='Android/obb/'). (edited)

Domi

Ah, and thinking about it : for this global / root of external storage access to be usefull, there is probably going to be something to do for the sync mechanism : with other rclone based solutions, sync from the root of external storage generates at least two access errors on Android/data and on Android/obb. Either it needs error handling or having the option to exclude some subdirectories from the sync (equivalent to --exclude='Android/data/' --exclude='Android/obb/'). (edited)

Since we've added support for filtering, if you don't mind tryingf feel free to use:

- Android/data/**
- Android/obb/**

or similar. minus character at the beginning of each line means exclude. More on that: https://rclone.org/filtering/ That would certainly be helpful for us, to come up with some sane "defaults". (edited)

Rclone Filtering

Rclone filtering, includes and excludes

Ah, sorry, I had missed the filtering option. Just tested it. Can't yet verify on /storage/emulated/0 or /storage/emulated/0/Android since they are not selectable until you add global storage access but how it works with other subdirectories should do it. On a Pixel phone, Android/data and Android/obb are the only two directories that register as errors during native rclone sync (leading rclone sync to not delete destination files, only do a kind of copy). (edited)

Tom

I've added this as an item for our team to check. We've used directory picker to stay compliant (and per directory permission model) with Android SDK's, however we've recently been given broader permissions, which may possibly allow us to select root location as well. (edited)

Hi Tom, juste wanted to check about this one. Being unable to select the root of the storage media (and of the Download directory by the way) is quite a limitation for doing full device backup. At the time being I do that with a rclone command in termux run through the tasker extension of termux and MacroDroid. Not really efficient and robust.

Hi @Domi, Sorry for the delay. I've made a note, so we can check that out next week. Hopefully we can make some amendments as it's clear from your message that in principle it should work just fine. Thanks!

1

Tom

Hi @Domi, Sorry for the delay. I've made a note, so we can check that out next week. Hopefully we can make some amendments as it's clear from your message that in principle it should work just fine. Thanks!

Thanks, and I don't want to make that super urgent. Definitely problematic for my use case but I don't know for others.

Domi

Thanks, and I don't want to make that super urgent. Definitely problematic for my use case but I don't know for others.

At the time being I do that with a rclone command in termux run through the tasker extension of termux and MacroDroid. Not really efficient and robust.

Quick question regarding your use case and temporary solution you seem to be using. Do you need root access on your phone? Some paths inside: /storage/emulated/0 doesn't seem to be accessible by other apps and I am not sure if OS gives any API to request permission for some of these directiories. In such case sync/copy for global path would possibly work once some of these directories are excluded. Do you perhaps already use some exclude rule for that use case?

I use that on a non rooted Pixel phone (and several other older tablets and phones). Termux (the one from f-droid not play store) has access to /storage/emulated/0 with the exception of Android/data and Android/obb. Other tools I used to sync before going the rclone route had the same access and limitations (for example Autosync/MetaCtrl). Anyway those two dirs are the only ones I exclude in my rclone command. The reason I asked you about exclusions some time ago

1:30 PM

To be precise, the command : rclone sync -v "/sdcard/" "**" --create-empty-src-dirs --exclude='Android/data/' --exclude='Android/obb/' 2>&1 | tee -a ~/rclone.log

Fair enough, thanks for sharing. We shall craft something along the lines over the next week.

1:36 PM

Is: /sdcard/* to match the identifier of the SD-CARD? On my end I see it like: /storage/1919-131B

Thanks. On most of the phones / tablets I have, /sdcard/ is an alias for /storage/emulated/0 not the external storage. I use it as a shortcut but also use /storage/emulated/0 depending on the mood

Hi, having seen that the latest version of the Android App tries to address this, I just tried the "custom" option that was added when creating a sync. I don't know how it is supposed to work but it does not seem completely operational. I tried again after having deleted / reinstalled the app to be sure, but with the same result. See screen capture. That on a Pixel 7, standard OS, no root, latest update. Hope it helps. Best regards. (edited)

1:15 PM

1

1:17 PM

Ah, and I'm ready to try test versions of the app if it can help.

Domi

Hi, having seen that the latest version of the Android App tries to address this, I just tried the "custom" option that was added when creating a sync. I don't know how it is supposed to work but it does not seem completely operational. I tried again after having deleted / reinstalled the app to be sure, but with the same result. See screen capture. That on a Pixel 7, standard OS, no root, latest update. Hope it helps. Best regards. (edited)

Thanks for your feedback. By any chance, can you visit About -> Logs and see if there is any related error? Prior to that you can also visit App settings (scroll to the bottom) and then enable: "Debug logs", as it will capture more entries.

Domi

Click to see attachment 🖼️

We've actually managed to reproduce the issue, however we need to find out the root cause of this issue. It doesn't seem to happen on all devices.

On the Pixel 7, I get this error : 2025-05-21 19:30:41.898 WARNING instance Failed to get Android paths. PlatformException(SDCardNotFound, No SD card available, null, null)

5:39 PM

On lineageos 21 / Android 14 on a Samsung tablet, I get the same result (gray widget) but no message in the logs. (edited)

On a Oneplus under Android 12, I have the same SDCardNotFound message in the logs. (edited)

6:19 PM

Both the Oneplus and Pixel don't have external storage / sdcard. The Samsung tablet has an sdcard. So maybe the message is not directly linked with this error message.

6:20 PM

Maybe you are using a widget that is not proposed on those versions of Android ? Samsung ? (edited)

6:21 PM

And from what I see, other solutions that don't use the system file picker because of the limitation on what can be selected tend to implement their own file / directory selector. Maybe a bigger task than it seemed at first. (edited)

@Domi Fix is awaiting release (Google should approve that hopefully today), there was a "widget" UI issue which it seems we haven't captured properly in Release mode. The SD card WARNING will stay for now, in the future it might be INFO instead. We try to get paths of phone storage (which is technically called external storage and equates in most cases to /storage/emulated/0), SD cards and and USB devices that you might have connected and want to obtain their paths. These buttons, e.g.: External storage or SD card or USB #1 etc. will prefill the custom path text field, but you're free to provide your own path. This tool will also prefill the exclude filter with some defaults. I am really keen to find out if this stuff works for you!. (edited)

That's great. I'll rush to test that as soon as it is published. And by the way, really impressed by your attention and reactivity! (edited)

Hi, we've finally got approved, so you should be able to see the newest version! (edited)

Hi, and yes ! I configured it yesterday evening and tried several things, then let it run during the night.

9:09 AM

The good news is that path selection works well. I was able to set up a full backup sync. Which is great !

9:10 AM

Still two hiccups.

9:14 AM

One is related to background sync : I've not completed tests about it but a) when the app is in the background it slows and fails to transfer bigger files, which leads to many retries and skipping to the next files, leaving incomplete copies online. It finally worked with leaving the app in the foreground, locking the phone, and leaving it plugged for the night. Still when the phone is locked and screen off the speed of copy falls rapidly.

9:15 AM

The result this morning is a complete sync : running a command line rclone sync shows nothing to do which is nice and means the sync was completed. (edited)

9:17 AM

But that leads to the second hiccup (sorry) : the sync status displays an error (edited)

9:17 AM

9:22 AM

9:23 AM

It seems that the exclusion of Android/obb/ (which is present in the exclusions list, and I tried both with relative and full path) does not work completly. Android/data/ seems to be taken care or correctly. (edited)

Led to this messages in the logs

9:41 AM

9:47 AM

Played with adding entries in the exclusions list after the two that were generated. Adding Android/media/ last had the effect of removing the distant media directory (expected), replacing the media line by a dummy tmp/ led to syncing back media correctly (and no error).

9:49 AM

More surprising, removing the tmp/ line from the exclusions, which should get back to a non working situation leads to a working one with correct syncing of the tmp directory (since it's not excluded anymore) and no obb errors.

9:51 AM

Maybe something with the default lines added to the exclusions list by the new custom option ? A training character or something ? Can't reproduce with just removing / recreating the sync conf. Will do more tests. (edited)

Couldn't exactly reproduce, but maybe an explanation : clean conf ; sync start, stopped during a large file sync then restarted which induces a lastError in the sync raw stats. It seems that this error is not cleared until the app is exited / force closed. Which leads to a recurring error each time the sync job is run again. The obb error may be a consequence of my attempts at finding the source of the error. (edited)

That would leave two problems : this incomplete cleanup of lastError and the background sync. On this one, it's systematic : the transfer stops approximatly 5 seconds after the app is put in the background by switching to another app. All requested permissions having been given. Media backup has the same problem. (edited)

Thanks for your feedback so far and taking time to test these things! It's extremely valuable for us, as it allows us jump to work straight away and deliver next series of improvements. I will get back to you once we process the feedback and/or we have some update regarding these aspects. It's our priority to get this working and (other items e.g. background), so please expect update rather sooner than later.

Happy if I can help. I really appreciate how you are handling things. And the app too

Hi @Domi, sorry for late reply. In recent releases we've slightly improved sync UI. Previously, in some cases the data that was displayed (mostly execution times) was partially showing new run and previous run which could be confusing. We've made sure that when user sets "Custom path" then existing filters are preserved instead of overwritten. Such behaviour could also lead to some confusion. Regarding syncing in background. Once app lands in the taskbar (that is no longer in the foreground), then existing app proceses (including sync) will be killed according to the OS / battery settings, in your case that was 5s (if I read correctly). Once you click top right icon in the Sync tab it will guide you through disabling battery optimization settings which may help to extend time before app no longer in the foreground gets killed. Additionally, the same button enables background sync, which aims to run your sync jobs even if app isn't running (no longer in taskbar). The background process is scheduled every 16 minutes and will try to run any sync jobs that needs running at that time (according to "sync every" settings) Initially you could try toggle the "background" button (only to disable battery optimization settings) and toggle that button again to try out whether it takes any longer before S3Drive gets killed by OS. If it works for you, you could keep it like that, if it doesn't give a background sync a go (by keeping the button enabled - green). I hope that eventually we can get this to work as expected. (edited)

Hi @Tom, I had already enabled / disabled the sync button and the fact that the battery optimization was disabled did not change much when running from the sync tab (with the start button). Still does not change much in the latest version when starting the task with the start button. I spent some time testing the background sync, even if the foreground sync did not complete correctly, and indeed the background sync seems to work consistently and do a backup every 8 hours as configured, including for "long" syncs with many and large files to transfer. I seem to understand that you are doing that with a foreground service (thus the notification) which seems to be the preferred option for running long background tasks (and maybe starting them with a background worload) ? If that's the case and as the sync service seems to work well, why don't you use it including when running a sync from the sync tab with the start button ? That would permit for the manually launched sync to finish even if the app is not left in the foreground. Another point : when the sync is started from the start button, stopped before it's finished, and started again this way, it seems that the error (from stopping early) is still not cleared. Ah, and may be interesting to have, later in the future : a progress status in the notification ; some kind of journal of sync ? Also, not sure I've seen the notification again recently. Is it always displayed ? (edited)

Domi

Hi @Tom, I had already enabled / disabled the sync button and the fact that the battery optimization was disabled did not change much when running from the sync tab (with the start button). Still does not change much in the latest version when starting the task with the start button. I spent some time testing the background sync, even if the foreground sync did not complete correctly, and indeed the background sync seems to work consistently and do a backup every 8 hours as configured, including for "long" syncs with many and large files to transfer. I seem to understand that you are doing that with a foreground service (thus the notification) which seems to be the preferred option for running long background tasks (and maybe starting them with a background worload) ? If that's the case and as the sync service seems to work well, why don't you use it including when running a sync from the sync tab with the start button ? That would permit for the manually launched sync to finish even if the app is not left in the foreground. Another point : when the sync is started from the start button, stopped before it's finished, and started again this way, it seems that the error (from stopping early) is still not cleared. Ah, and may be interesting to have, later in the future : a progress status in the notification ; some kind of journal of sync ? Also, not sure I've seen the notification again recently. Is it always displayed ? (edited)

That's great to hear! Background sync is a relatively new feature, so to play safe (and make sure it doesn't interfere with the foreground sync) we've kept it separate. We still work on certain "status update" improvements to make it better integrated, for instance if background sync is running and you open S3Drive, then currently it won't show that sync is running even though that technically it is (in the background).

I seem to understand that you are doing that with a foreground service (thus the notification)

It's not a foreground service, it's a background service. Notification is explicitly added by us to indicate that there is some processing being done.

why don't you use it including when running a sync from the sync tab with the start button ?

Given feedback we receive we'll likely make it part of "Start" in one of the future releases.

a progress status in the notification

In a next 1.13.0 release we refactored functionality that gives the file transfer status and even though it's still far from great, it's better than current one (which hides all the data in Transfers tab). In the future we'll publish that progress status as a configurable notification.

some kind of journal of sync

That's a good idea, we'll certainly get to that hopefully soon. We could indicate failure, last full successful run and potentially next run datetime.

Domi

Hi @Tom, I had already enabled / disabled the sync button and the fact that the battery optimization was disabled did not change much when running from the sync tab (with the start button). Still does not change much in the latest version when starting the task with the start button. I spent some time testing the background sync, even if the foreground sync did not complete correctly, and indeed the background sync seems to work consistently and do a backup every 8 hours as configured, including for "long" syncs with many and large files to transfer. I seem to understand that you are doing that with a foreground service (thus the notification) which seems to be the preferred option for running long background tasks (and maybe starting them with a background worload) ? If that's the case and as the sync service seems to work well, why don't you use it including when running a sync from the sync tab with the start button ? That would permit for the manually launched sync to finish even if the app is not left in the foreground. Another point : when the sync is started from the start button, stopped before it's finished, and started again this way, it seems that the error (from stopping early) is still not cleared. Ah, and may be interesting to have, later in the future : a progress status in the notification ; some kind of journal of sync ? Also, not sure I've seen the notification again recently. Is it always displayed ? (edited)

Another point : when the sync is started from the start button, stopped before it's finished, and started again this way, it seems that the error (from stopping early) is still not cleared.

We clear errors or set a new one at the end of the run, so if you stopped sync (this chould throw an error) and restarted it, the previous error will persist until the execution of just started sync finishes. Unless I misunderstood it?

Tom

Another point : when the sync is started from the start button, stopped before it's finished, and started again this way, it seems that the error (from stopping early) is still not cleared.

We clear errors or set a new one at the end of the run, so if you stopped sync (this chould throw an error) and restarted it, the previous error will persist until the execution of just started sync finishes. Unless I misunderstood it?

for example, if I start a sync with a file to transfer and stop it after a few seconds (before the file sync was completed), there is a lastError "Put... " : context canceled" ; if I restart the sync job, the lasterror does not seem to be cleared and if I let this job finish, the last error is still there, and the failed count is incremented (failed 2 times) although the job ran well. If I stop and start it again (with nothing much to do since everything is synced), I still have an error and failed count is incremented again. (edited)

10:58 AM

Anyway, I'm quite positive with what you have done : background sync indeed works reliably.

1

Domi

for example, if I start a sync with a file to transfer and stop it after a few seconds (before the file sync was completed), there is a lastError "Put... " : context canceled" ; if I restart the sync job, the lasterror does not seem to be cleared and if I let this job finish, the last error is still there, and the failed count is incremented (failed 2 times) although the job ran well. If I stop and start it again (with nothing much to do since everything is synced), I still have an error and failed count is incremented again. (edited)

I see, I am going to try to reproduce this issue and if I can, then I shall have better understanding why is it happening. (edited)