Encryption is really slow, considering encfs

[SanskritFritz]

Hi, first post here, new obnam user, have lingered for a while on IRC #obnam.

I'm testing obnam, and find it great. However when I try to encrypt my
backup, it becomes really slow.
http://blog.liw.fi/posts/obnam-localfs-speed/ confirms this.
encrypted:
Backed up 18459 files (of 18459 found), uploaded 796.0 MiB in 2h20m54s
at 96.5 KiB/s average speed
not encrypted:
Backed up 18459 files (of 18459 found), uploaded 796.0 MiB in 2m19s at
5.7 MiB/s average speed
Of course there was some caching I guess, but the difference is
striking and running obnam days ago for the first time (without
encryption) was fast already.

My plan is to perform local backups and lftp mirror the repository to
an untrusted server, so encyption is a must for me, but not locally,
since my harddisk is encrypted already. Given the enormous speed
difference, I'm playing with the idea of performing unencrypted
backups and expose the repository through "encfs --reverse "to the
server. The mirroring would be incremental, given that the file names
in the repository do not change, which I assume.

Is that solution feasible? Encfs is stable enough for me, I have
mirrored my pictures collection this way to the server for years now.
Or is there a way to speed up the encryption in obnam that I don't
know about?

Thanks in advance.
SanskritFritz

[Lars Kruse]

Hi,

Post by SanskritFritz
My plan is to perform local backups and lftp mirror the repository to
an untrusted server, so encyption is a must for me
[..]
Is that solution feasible?

I am using just the setup that you described for about one year now. It did not
expose any visible problems for now.
Attached you find the script that I use for the mount/rsync/umount cronjob.
Maybe it is useful for you.

Post by SanskritFritz
Or is there a way to speed up the encryption in obnam that I don't
know about?

As far as I understand the current implementation, a gnupg process is created
for every single encryption/decryption request. Probably this could be improved
by re-using one instance of gnupg for multiple requests (maybe via 'command-fd'
and 'status-fd'). Alternatively python libraries could be used for encryption.
The current approach (as far as I understand Lars' motivation) is supposed to
minimize the risk of mistakes/faults with regard to encryption by using a
widespread standard tool with its default setup.

Cheers,
Lars K.

[SanskritFritz]

Post by Lars Kruse

Post by SanskritFritz
My plan is to perform local backups and lftp mirror the repository to
an untrusted server, so encyption is a must for me
[..]
Is that solution feasible?

I am using just the setup that you described for about one year now. It did not
expose any visible problems for now.
Attached you find the script that I use for the mount/rsync/umount cronjob.
Maybe it is useful for you.

Thanks for the script, it looks really useful for me, I will
definitely use the ideas therein.
So, good to know, I'm not alone and that my planned workaround could
work out. Thanks for your answer.

[SanskritFritz]

Hi, first post here, new obnam user, have lingered for a while on IRC #obnam.

I'm still testing obnam, and find it great. However when I try to
encrypt my backup, it becomes really slow.
http://blog.liw.fi/posts/obnam-localfs-speed/ confirms this.
encrypted:
Backed up 18459 files (of 18459 found), uploaded 796.0 MiB in 2h20m54s
at 96.5 KiB/s average speed
not encrypted:
Backed up 18459 files (of 18459 found), uploaded 796.0 MiB in 2m19s at
5.7 MiB/s average speed
Of course there was some caching I guess, but the difference is
striking and running obnam days ago for the first time (without
encryption) was fast already.

My plan is to perform local backups and lftp mirror the repository to
an untrusted server, so encyption is a must for me, but not locally,
since my harddisk is encrypted already.
Given the enormous speed difference, I'm playing with the idea of
performing unencrypted backups and expose the repository through
"encfs --reverse "to the server. The mirroring would be incremental,
given that the file names in the repository do not change, which I
assume.

Is that solution feasible? Encfs is stable enough for me, I have
mirrored my pictures collection this way to the server for years now.
Or is there a way to speed up the encryption in obnam that I don't know about?

Thanks in advance.
SanskritFritz