script to nuke items from th-archive that are already in snap

objective for such cleanup is to decrease number of packages in th-archive:
1. faster loading of index
2. poldek hardcoded 64k limit of packages per repo

before:

source packages in th-archive: 20120

estimated[1] packages to remove:
2012: 3514 packages
2013: 3996 packages
2014: 4452 packages
2015: 3279 packages

after:

packages removed from th-archive that were present in snap:
2012: 3520 packages
2013: 2493 packages
2014: 2569 packages
2015: 2443 packages

source packages in th-archive: 9451

[1] does not account that package can be also in earlier snap

1 files changed, 38 insertions, 0 deletions

diff --git a/wwwbin/archive-dupe-cleanup.sh b/wwwbin/archive-dupe-cleanup.sh
new file mode 100755
index 0000000..e00ad7a
--- /dev/null
+++ b/wwwbin/archive-dupe-cleanup.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+set -e
+
+archive_tree=.archive/PLD
+other_tree=$1
+ftp_dir=~ftp/pld/dists/th
+
+die() {
+	echo >&2 "$*"
+	exit 1
+}
+
+
+test -n "$other_tree" || die 'specify other tree, ex: 2015/PLD'
+
+archive_dir=$ftp_dir/$archive_tree/SRPMS/.metadata
+other_dir=$ftp_dir/$other_tree/SRPMS/.metadata
+
+test -d "$archive_dir" || die '$archive_dir not a dir'
+test -d "$other_dir" || die '$other_dir not a dir'
+
+match_pkgs() {
+	for f in $archive_dir/*.info; do
+		fn=${f##*/}
+		if [ -f "$other_dir/$fn" ]; then
+			echo $fn
+		fi
+	done
+}
+
+remove_pkgs=$(match_pkgs)
+test -n "$remove_pkgs" || die "nothing matched"
+c=$(echo "$remove_pkgs" | wc -l)
+echo >&2 "$c packages to remove from $archive_dir. ctrl-c to abort"
+read a
+
+echo >&2 pfa-rmpkg $archive_tree $remove_pkgs
+pfa-rmpkg $archive_tree $remove_pkgs