How long should committing to repository take?

[Mario]

Hallo everybody,

I have a short question: How long should committing to repository take?
I have around 1 TB of HOME to backup. Last time I added about 300 GB to
the repo. The scanning took around 6 hours or so.
So the last entry in the log file was "Backing up /home/mario/".

After this Obnam run for more than 9 hours. I eventually killed it,
because I had to switch to Windows.
Is this a usual behavior? To be honest I do not fully understand what
Obnam does, just that it splits up the files in chunks
and fills them to some repo. What happens exactly in the last step?

Cheers


Mario

[Mario]

Hallo everybody,

I have a short question: How long should committing to repository take?
I have around 1 TB of HOME to backup. Last time I added about 300 GB to
the repo. The scanning took around 6 hours or so.
So the last entry in the log file was "Backing up /home/mario/".

After this Obnam run for more than 9 hours. I eventually killed it,
because I had to switch to Windows.
Is this a usual behavior? To be honest I do not fully understand what
Obnam does, just that it splits up the files in chunks
and fills them to some repo. What happens exactly in the last step?

Cheers


Mario

[Mario]

I should update my question (I was at my office, when I wrote the last
mail).
So the part where obnam is taking a lot of time is not "committing to
repo" but
"files 704.92 GiB scanned: removing checkpoints".

It consumes CPU like hell, but in the logs (with --level=DEBUG) I see
nothing
happen.

Sry for my first misleading mail (and for double post).

Cheers

Mario

Post by Mario
Hallo everybody,
I have a short question: How long should committing to repository take?
I have around 1 TB of HOME to backup. Last time I added about 300 GB
to the repo. The scanning took around 6 hours or so.
So the last entry in the log file was "Backing up /home/mario/".
After this Obnam run for more than 9 hours. I eventually killed it,
because I had to switch to Windows.
Is this a usual behavior? To be honest I do not fully understand what
Obnam does, just that it splits up the files in chunks
and fills them to some repo. What happens exactly in the last step?
Cheers
Mario

[Lars Wirzenius]

In the checkpoint removal phase, Obnam removes all the checkpoint
generations it makes during a backup. Checkpoint generations are made,
by default, every one gigabyte of backed up data, and their purpose is
to avoid having to re-backup things if there's a crash. For example,
if you are backing up 300 gigabytes of new data, and the computer
crashes after 299 gigabytes, without checkpoint generations the next
run of Obnam would backup all 299 gigabytes again. With checkpoints,
it only backs up again the last gigabyte.

Unfortunately, removing many generations can be slow with Obnam, so
the checkpoint generation removal can take quite excessively long
times, as you've seen. You can avoid that by either making the
checkpoint interval be much larger, or use the --leave-checkpoints
option, which prevents them from being removed.

Checkpoint generations can be removed later with "obnam forget", just
like any other generations.

The "committing to repo" phase can also take a long time, and progress
reporting with that is (also) not very good. In the commit phase,
Obnam is moving files around in the repository. This is due to the way
Obnam implements crash safety: it writes all changes first to a
"journal", and when all changes have been written to the journal,
including a marker that the journal is complete, it moves them to
their right places. If Obnam crashes before the completion marker is
written, the next run of Obnam will clear the journal: it's as if the
crashed run of Obnam never happened. If Obnam crashes after the
completion marker is written, but before all files have been moved
from the journal to their rightful place, the next run of Obnam will
complete the moving of files.

I hope that clarifies.

Post by Mario
I should update my question (I was at my office, when I wrote the
last mail).
So the part where obnam is taking a lot of time is not "committing
to repo" but
"files 704.92 GiB scanned: removing checkpoints".
It consumes CPU like hell, but in the logs (with --level=DEBUG) I
see nothing
happen.
Sry for my first misleading mail (and for double post).
Cheers
Mario

Post by Mario
Hallo everybody,
I have a short question: How long should committing to repository take?
I have around 1 TB of HOME to backup. Last time I added about 300 GB
to the repo. The scanning took around 6 hours or so.
So the last entry in the log file was "Backing up /home/mario/".
After this Obnam run for more than 9 hours. I eventually killed it,
because I had to switch to Windows.
Is this a usual behavior? To be honest I do not fully understand what
Obnam does, just that it splits up the files in chunks
and fills them to some repo. What happens exactly in the last step?
Cheers
Mario

_______________________________________________
obnam-support mailing list
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-support-obnam.org