Discussion:
[PATCH] git-svn: clear global SVN pool between get_log invocations
Eric Wong
2014-10-21 03:39:12 UTC
Permalink
During long fetches with many revisions, some SVN functions are not
always using local pools and some gets into the global pool. Ensure
we free that memory occasionally between SVN::RA::get_log calls to
reduce memory growth.

This reduces memory usage over the course of fetching 10K revisions
using a test repository created with the script at the end of this
commit message.

As reported by time(1) on my x86-64 system:

before: 54024k
after: 47344k

-----------------------------8<------------------------------
set -e
tmp=$(mktemp -d svntestrepo-XXXXXXXX)
svnadmin create "$tmp"
repo=file://"$(cd $tmp && pwd)"
svn co "$repo" "$tmp/wd"
cd "$tmp/wd"
if ! test -f a
then
a
svn add a
svn commit -m 'A'
fi

nr=10000
while test $nr -gt 0
do
echo $nr > a
svn commit -q -m A
nr=$((nr - 1))
done
echo "repository created in $repo"
-----------------------------8<------------------------------

Signed-off-by: Eric Wong <***@yhbt.net>
---
Pushed to master of git://bogomips.org/git-svn

However, memory usage still seems to grow infinitely even in this simple
fetch case, so are other memory leaks.

perl/Git/SVN/Ra.pm | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index a7b0119..3df4591 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -237,10 +237,6 @@ sub trees_match {
my $ctx = SVN::Client->new(auth => _auth_providers);
my $out = IO::File->new_tmpfile;

- # older SVN (1.1.x) doesn't take $pool as the last parameter for
- # $ctx->diff(), so we'll create a default one
- my $pool = SVN::Pool->new_default_sub;
-
$ra_invalid = 1; # this will open a new SVN::Ra connection to $url1
$ctx->diff([], $url1, $rev1, $url2, $rev2, 1, 1, 0, $out, $out);
$out->flush;
@@ -453,6 +449,7 @@ sub gs_fetch_loop_common {
$_[0] = undef;
$self = undef;
$RA = undef;
+ SVN::Core->gpool->clear;
$self = Git::SVN::Ra->new($ra_url);
$ra_invalid = undef;
}
@@ -468,6 +465,7 @@ sub gs_fetch_loop_common {
my $k = "svn-remote.$g->{remote}.$g->{t}-maxRev";
Git::SVN::tmp_config($k, $max);
}
+ SVN::Core->gpool->clear;
last if $max >= $head;
$min = $max + 1;
$max += $inc;
--
EW
--
EW
Loading...