S3Drive - general

7:10 PM

Any idea how to fix this? Encryption is enabled

Hi @exploit, thanks for your post. I assume that's an Android? Can I ask you what's your OS version and phone brand/model? Does it happen for every file during upload? (edited)

Tom

Hi @exploit, thanks for your post. I assume that's an Android? Can I ask you what's your OS version and phone brand/model? Does it happen for every file during upload? (edited)

Galaxy S21 Ultra, Android 13 (edited)

10:30 AM

Yeah since I enabled encryption

@exploit Thanks, we're fully on this. We've managed to reproduce it on some low-end Samsung with Android 13. I will let you know later today about progress on this. Sorry for the inconvenience !

Tom

@exploit Thanks, we're fully on this. We've managed to reproduce it on some low-end Samsung with Android 13. I will let you know later today about progress on this. Sorry for the inconvenience !

No worries, I was just trying the app for the first time

(edited)

1

exploit

No worries, I was just trying the app for the first time

(edited)

If you come across something else that's not working for you or it's not clear enough, I'd be more than happy to help.

1

exploit

Yeah since I enabled encryption

We've addressed this issue in the 1.1.3 release which is currently pending review on the Play Store. I will let you know once it's Live (this usually takes between ~5-12 hours). Thank you very much for reporting this.

exploit

Any idea how to fix this? Encryption is enabled

This issue is now fixed, please update your app version. Thanks ! (edited)

2

FortyMarlin joined the server. 3/13/2023 11:48 PM

Hi @FortyMarlin, welcome to our little world !

1

Keep up the great work Tom! This is turning into a better app every week!

1

How do i support this app? Is it for free?

shaike

How do i support this app? Is it for free?

App is entirely free. You're supporting us by simply using it, as you would likely report an issue, or perhaps might have some improvement idea. We're working on a sustainable business model, but at the moment we're focused on delivering remaining functionalities and moving it from (beta) to (stable). We'll be offering managed S3 accounts in Q2, but will always leave an option for a user to use their own S3 account. (edited)

Thanks much

shaike

Thanks much

If you like the app, we would appreciate rating our app and/or leaving the review on App/Play Store. I think the idea behind the app is great and fits many uses cases, but we're pretty early and far from being a recognizable solution

Logarithm joined the server. 3/16/2023 12:41 AM

I use StorJ but found that upload/download speeds can sometimes be abyssimal and requests to delete folders with many thousand items fail for no good reasons. I'm considering to move my data to Wasabi, thoughts?

Logarithm

I use StorJ but found that upload/download speeds can sometimes be abyssimal and requests to delete folders with many thousand items fail for no good reasons. I'm considering to move my data to Wasabi, thoughts?

@Logarithm Hi, thanks for joining. There are couple items here. Do you have an encryption enabled and use files bigger than 100MB? If so, then S3Drive uses much slower software encryption for such files which has visible performance effects, we plan to fix this, however since it's a topic of highly security sensitive nature, we're not rushing the decision. These are the roadmap items: https://s3drive.canny.io/feature-requests/p/explore-support-for-xchacha20-poly1305 and https://s3drive.canny.io/feature-requests/p/support-streaming-of-encrypted-videos which we plan to deliver around Summer 2023. If you upload/download multiple items without encryption enabled or smaller than 100MB and find poor upload/download speeds then switching to different S3 (Wasabi) might help, but only if you're not already maxing out your current internet connection. Deleting item is a different beast. Currently we're not deleting items, but rather "copy" them to trash one by one, and then delete files in the original location. It's not that delete is slow, but the "copy" operation takes time and it's safe to assume that "thousand" files operation will be roughly 1000x slower then a single file. What can help here is to switch to S3 provider/region that has closest proximity to your internet connection, but no matter what results won't be satisfying enough. No worries though, we plan to address this in ~April 2023 by adding support for native S3 versioning: https://s3drive.canny.io/feature-requests/p/version-preview-restore-integration-with-trash so we can actually skip Copy operation... and Delete itself is fast enough. (edited)

1

Logarithm

I use StorJ but found that upload/download speeds can sometimes be abyssimal and requests to delete folders with many thousand items fail for no good reasons. I'm considering to move my data to Wasabi, thoughts?

I've just checked it and it seems that Storj doesn't support object versioning and I didn't find if it's on their radar to have it implemented. On the other hand Wasabi does support it: https://wasabi-support.zendesk.com/hc/en-us/articles/115001684631-Does-Wasabi-support-versioning-

Does Wasabi support versioning?

Yes. The versioning feature with a cloud storage service offers an additional level of protection by providing a means of recovery when customers accidentally overwrite or delete objects. This allo...

1

Thanks mate!

I left a 5 stars review for both apps hope it helps it rank high in search results

1

3:36 PM

What is the UI coded in? React native or flutter?

Hi, which S3 storage would you recommend to connect to?

FortyMarlin

Hi, which S3 storage would you recommend to connect to?

Hi, They all have slightly different pricing models, each of them being suited towards certain workflows. You probably can't go wrong with either Backblaze (PayG but transfer fees) or Wasabi (fixed plan, but no transfer fees). I didn't have technical problems with Scaleway (except some billing issues and requirement of using credit card instead of debit card). If you want fully "own" the data, then you can self-host MinIO, but this would only be cost-effective for bigger datasets or if you can afford setting it up on at your Home/NAS. I probably wouldn't recommend iDrive E2 (lots of technical issues and inconsistent behavior)... and with Storj I need to say I don't exactly know what guarantees they give, but you probably can't go wrong with free ~~~150GB~~ (UPDATE: They've changed that to 25GB - https://forum.storj.io/t/storj-free-tier-update-for-new-accounts-25-gb-storage/22155) data plan just to try them out. No matter what, I wouldn't recommend using just one destination as the only backup of your data, but similar applies to Dropbox and alike. (edited)

1

Logarithm

I left a 5 stars review for both apps hope it helps it rank high in search results

Thank you, it always helps !

Logarithm

What is the UI coded in? React native or flutter?

We're using Flutter and it works really well. All of the logic, networking, encryption is also built using Flutter / Dart... and if something can't be achieved with Flutter, then we call native platform functions. (edited)

1

Great choice

Logarithm

Great choice

Are you interested in a cross-platform development? There is also other promising project: https://kotlinlang.org/docs/multiplatform.html it wasn't mature enough when we started. EDIT: Apparently it's still in Beta. (edited)

I have plans to work on kids mobile games but will probably orchestrate instead of code. I have some coding background but never loved coding much.

Logarithm

I have plans to work on kids mobile games but will probably orchestrate instead of code. I have some coding background but never loved coding much.

Well, I never loved coding as an isolated activity, but I find it extremely exciting when used as a tool to solve some real life problems. Good luck on your project ! (edited)

1

@Tom you think that Kotlin Multiplatform is mature enough to match Flutter? I couldn't find performance comparisons. It seems like Kotlin would be a good choice as it's a first class citizen for android too so you don't have to learn as many programming languages, but do they have rendering engines as good as flutter? I know flutter just released a new rendering engine for IOS that use metal, and their android engine is pretty good too.

Logarithm

@Tom you think that Kotlin Multiplatform is mature enough to match Flutter? I couldn't find performance comparisons. It seems like Kotlin would be a good choice as it's a first class citizen for android too so you don't have to learn as many programming languages, but do they have rendering engines as good as flutter? I know flutter just released a new rendering engine for IOS that use metal, and their android engine is pretty good too.

I don't think I am educated enough on Kotlin Multiplatform to answer that. I don't think that currently KMM maturity is near enough of Flutter, but specifics probably depends on the project. I think Kotlin has a better future than Dart though. I also liked working with Kotlin more... ~~and the ecosystem is Java compatible, which means you have access to decades of knowledge and different projects.~~ EDIT: It's not exactly that, Kotlin is apparently going into "language on it's own" direction and not necessarily aiming to use JVM (source: https://stackoverflow.com/a/74957737). This slightly alters my argument, still I've seen plenty of tools to migrate Java to Kotlin if you ever needed to port some Java logic. Yes, Flutter recently allows enabling Impeller engine on iOS and Android, previously Skia was used (I am actually not sure if it's not the same engine that KMM uses). To be honest it's not even about Kotlin vs Dart. If you know one language you can pretty easily use another. It's about what libraries, packages you get out of the box in one ecosystem or the other - https://pub.dev/ I think that Flutter ecosystem it's still early, but mature enough for many projects. . Bear in mind that you very often rely on some library which gives you native access to certain platform. These are not quick to build yourself and save you lots of time if you find one. I don't think KMM has many of these libraries, so often you will have to build one yourself in Kotlin and connect it to Swift and Android SDK (in which case you need to know the native language as well) (edited)

1

I wonder if Microsoft won't just catch up with their MAUI and they have lots of addons and packages etc... dotnet is pretty solid. I think I prefer the way you can declare your ui in XML and then mode the interactions or something. Not sure yet. Performance probably sucks big time however.

1:51 AM

I jist hate the JVM so much, that and gradle or maven or we... everything Java I dislike.

1:53 AM

Not sure it matters much which stack you pick today as they will all evolve differently so I'd go with the one that feels right to begin with and has the documentation and tools do that I want to do.

Daniel_Recker_95 joined the server. 3/20/2023 2:13 AM

Logarithm

I wonder if Microsoft won't just catch up with their MAUI and they have lots of addons and packages etc... dotnet is pretty solid. I think I prefer the way you can declare your ui in XML and then mode the interactions or something. Not sure yet. Performance probably sucks big time however.

Never had chance to work with C#, but it seems is pretty robust, mature language and for quite a while no longer locked in to Microsoft. My understanding is that Xamarin or MAUI they work in a similar way as React Native, which is you still call native UI components. (On the contrary Flutter uses the rendering engine)... so I don't think that performance will be that badly affected. Speaking of XML UI, I worked with Android and really didn't enjoy it (perhaps the MAUI one is nicer?), but before I tried Jetpack Compose I've introduced Flutter. The nested / React way of defining components is much clearer to me than XML. I never liked Java, it's verboseness, clunkiness and being memory hungry. With Android development I was forced to try it... and Kotlin made the introduction much nicer. After a while Gradle is no longer an issue, as you know the quirks. There are exciting projects like GraalVM which make resource startup costs of Java negligible. You can then reuse your project language on the back-end with huge choice of Reactive back-end frameworks (so far tried Vert.x and liked it). I was amazed by understanding how portable the Java ecosystem is... also I've fixed my previous comment and be aware that Kotlin (at least in KMM) is taking: "language on it's own" path and stepping away from JVM. One thing I've understood a while ago, perfect is the enemy of good. I better start using some technology and learn from it, the constantly try to find the best language / ecosystem, which may actually not exist. Same applies to Flutter, my biggest concern is that Google will simply kill it (https://killedbygoogle.com/), but before that happen I am sure that either there will be nice alternative and/or I will have time&resources to migrate my project elsewhere. After all it's Flutter that initially saved me a lot of time, so if I need to "pay it back" a little later, that's still fine with me. (edited)

Hi @Daniel_Recker_95 , welcome to the channel !

polyenka joined the server. 3/20/2023 2:03 PM

Hi @polyenka, welcome to our channel !

pairg joined the server. 3/22/2023 8:55 PM

Hi @pairg, welcome here ! (edited)

zer0 joined the server. 3/24/2023 5:23 AM

Hi @zer0 , welcome to our small community !

zer0

Click to see original message

8:03 AM

Interesting software

1

... and it will only get even more interesting with our ambitious 2023 Roadmap - https://s3drive.canny.io

Nice

8:10 AM

I'm a homelabber so I'd love to host this on my server someday

8:12 AM

If you don't mind me asking, is Sync Solutions a company?

zer0

I'm a homelabber so I'd love to host this on my server someday

The architecture of the client does require valid S3 endpoint. You can run e.g. MinIO on your end: https://hub.docker.com/r/minio/minio/#! which gives fully S3 compatible API. In such setup it's only S3Drive client on your device hitting your MinIO endpoint which is also in your control.

Tom

The architecture of the client does require valid S3 endpoint. You can run e.g. MinIO on your end: https://hub.docker.com/r/minio/minio/#! which gives fully S3 compatible API. In such setup it's only S3Drive client on your device hitting your MinIO endpoint which is also in your control.

Is this also true for the webapp when used with local s3 servers? Or is data proxied through another server?

zer0

If you don't mind me asking, is Sync Solutions a company?

It's a trade name, we're currently registered as a sole proprietorship, we're pending registration and paperwork to switch to limited company. This will likely be finished around Q2/Q3 2023. It is possible the trade name will change at that point. (edited)

Cool. Nothing wrong with companies doing FOSS. Just trying to understand the motivations and goals

zer0

Is this also true for the webapp when used with local s3 servers? Or is data proxied through another server?

Webapp is exactly the same codebase as desktop and mobile clients (thanks to Flutter), so it's hitting the API directly, you can verify that in your browser (right-click + inspect element and see the network tab - XHR requests). There are few exceptions where we might hit some other server (but never for S3 related operations), but we've mentioned that explicitly in our privacy section: https://s3drive.app/privacy

Hmm

8:22 AM

I see pro features mentioned. Do you see a bitwarden-like business model?

zer0

Cool. Nothing wrong with companies doing FOSS. Just trying to understand the motivations and goals

Just for you to be aware we're not exactly FOSS. In our setup the clients code is closed, but the back-end protocol (S3) is open, meaning that you can always switch to other S3 client if you're not happy. There are other models on the market. For instance there are multiple companies having their clients open-sourced, but back-end is closed, which has some other advantages, but also disadvantages (vendor lock-in being one of them).

Oooh

8:25 AM

Yea

8:25 AM

Ok I skimmed the webpage and thought it was FOSS

8:27 AM

So files can be decrypted with or without s3drive am I understanding this correctly?

8:28 AM

Both the encryption and backend are safe from vendor lock-in?

zer0

I see pro features mentioned. Do you see a bitwarden-like business model?

We've been thinking about sustainable business model for a while. Our plans include hosted S3 accounts, which would include all of the Pro features when you get the data plan. BItwarden business model, we're not there yet, maybe eventually.

zer0

So files can be decrypted with or without s3drive am I understanding this correctly?

That's right, this is because we're compatible with the AWS FOSS implementation: https://docs.aws.amazon.com/general/latest/gr/aws_sdk_cryptography.html#crypto_features

That's great. That makes me feel more confident

zer0

That's great. That makes me feel more confident

In our roadmap we plan to release open source decryption / export tool: https://s3drive.canny.io/feature-requests/p/open-source-data-decryption-export-tool

Very nice. Reminds me of standard notes

zer0

Both the encryption and backend are safe from vendor lock-in?

Back-end yes, no vendor lock-in, because we plan to stay compatible with S3. There maybe additional extra features on top of S3 protocol which are not possible to achieve with current S3 (related to e.g. file sharing and atomic rename/copy) in which case we might come up with some optional middle-man.

So far I see this as a valid replacement to rclone WebDAV serve with rclone's crypt -- for less technical people who are afraid of CLIs

zer0

Both the encryption and backend are safe from vendor lock-in?

Regarding encryption, there are couple challenges that we face with current AES-256 GCM implementation (mostly related to bigger files and video streams). We have plans to improve that: https://s3drive.canny.io/feature-requests/p/explore-support-for-xchacha20-poly1305 https://s3drive.canny.io/feature-requests/p/support-streaming-of-encrypted-videos but that still needs more research and there are couple dependencies. No matter what, our encryption standard will remain open and if possible compatible with other tooling.

zer0

So far I see this as a valid replacement to rclone WebDAV serve with rclone's crypt -- for less technical people who are afraid of CLIs

Yes, but hopefully not only this. I find S3Drive way more convenient tool than rclone for accessing my files or backing up my photos on a day to day basis.... and I am not afraid of CLIs. (edited)

yes that too of course

8:39 AM

S3 compatible app on android at least, that looks nice is rare

8:39 AM

Even if I don't use encryption, it's still handy to manage a bucket with a GUI

zer0

S3 compatible app on android at least, that looks nice is rare

Heh, thanks for that. Our design is far from modern expectations, but well, there are other S3 clients which kind of lowered entry bar for us

design is not bad at all

8:40 AM

i kinda like it

8:41 AM

on desktop tho, it feels very weird to use a smartphone/tablet-ish UI

zer0

on desktop tho, it feels very weird to use a smartphone/tablet-ish UI

You're right, on the plus side, this allowed us to release desktop clients early. GUI will be improved, once we get through other items... and we're fully booked for 2023 so far.

also the taskbar icon imho looks too close to windows explorer icon

zer0

also the taskbar icon imho looks too close to windows explorer icon

That's a valid point, thanks. Being an Ubuntu user myself didn't have that graphic encoded in my mind.

@zer0 If you have any other questions, concerns I would be glad to help out... and speaking of FOSS you might actually think of reasons behind our decision and it's mostly related to the fact that we try to build a sustainable business. With our open back-end model, OSS clients would simply kill any ways for us to monetize our work. OSS in general has funding issues and there are many examples of that. Companies in order to sell OSS, try to build closed source back-end, vendor lock-in you into their solution and make self-hosting pretty hard (despite code being open source). It's fine to find ways to monetize their work, it's just it's little bit deceptive practice (to claim Free, Open, but being far from it) in my opinion. As with every business, things may or may not work out. We plan to introduce certain safeguard triggers (e.g. if we ever planned to wind down our operations we would then release our work on some open-source license), more on that in our next updates. (edited)

of course

9:00 AM

all the best

9:01 AM

oh something you might want to look into:

9:01 AM

i use this https://garagehq.deuxfleurs.fr/ at home as my s3 server

Garage - An open-source distributed object storage se...

An open-source distributed storage service you can self-host to fullfill many needs.

1

9:01 AM

s3drive doesn't seem to work with it

9:02 AM

works fine with rclone, arq, cloudberry just fine

9:03 AM

zer0

Click to see attachment 🖼️

Thanks, we'll look into that.

zer0

Click to see attachment 🖼️

I know I probably ask for too much, but do you have an example endpoint I could use to try it out? If not, no worries we'll have it running and then play around.

sure

9:06 AM

allow me an hour

1

zer0

allow me an hour

Great, thanks that would speed things up on our end. Don't feel rushed though. We're here most of the day anyway. (edited)

anytime

zer0

Click to see attachment 🖼️

I did a quick setup on my local: https://garagehq.deuxfleurs.fr/documentation/quick-start/ and discovered that the issue is that by default in enforces region: garage... and there is no way to provide region in S3Drive. It seems that we may add additional form field to specify the region.

zer0

Click to see attachment 🖼️

The quick fix if you want to use S3Drive is to set the region in toml file like this: s3_region = "us-east-1". We auto-detect region from the endpoint URL and have a way to detect custom region from MinIO.... and if it doesn't work we use the most common default which is us-east-1.