* [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/
@ 2019-11-28 16:34 Joonas Niilola
0 siblings, 0 replies; 5+ messages in thread
From: Joonas Niilola @ 2019-11-28 16:34 UTC (permalink / raw
To: gentoo-commits
commit: b2a8f934671d15ac008f42976e5c4de087e31607
Author: Kai Krakow <hurikhan77+bgo <AT> gmail <DOT> com>
AuthorDate: Thu Nov 28 11:23:02 2019 +0000
Commit: Joonas Niilola <juippis <AT> gentoo <DOT> org>
CommitDate: Thu Nov 28 16:34:35 2019 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b2a8f934
sys-fs/bees: Bump to v0.6.1-r1
Closes: https://bugs.gentoo.org/685714
Package-Manager: Portage-2.3.76, Repoman-2.3.16
Signed-off-by: Kai Krakow <hurikhan77+bgo <AT> gmail.com>
Signed-off-by: Joonas Niilola <juippis <AT> gentoo.org>
sys-fs/bees/bees-0.6.1-r1.ebuild | 86 ++++++++++++++++++++
...-local-lambda-expression-cannot-have-a-ca.patch | 38 +++++++++
...karound-to-prevent-LOGICAL_INO-and-btrfs-.patch | 95 ++++++++++++++++++++++
3 files changed, 219 insertions(+)
diff --git a/sys-fs/bees/bees-0.6.1-r1.ebuild b/sys-fs/bees/bees-0.6.1-r1.ebuild
new file mode 100644
index 00000000000..5d44fcd0182
--- /dev/null
+++ b/sys-fs/bees/bees-0.6.1-r1.ebuild
@@ -0,0 +1,86 @@
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=7
+
+inherit linux-info systemd
+
+DESCRIPTION="Best-Effort Extent-Same, a btrfs dedup agent"
+HOMEPAGE="https://github.com/Zygo/bees"
+
+if [[ ${PV} == "9999" ]] ; then
+ EGIT_REPO_URI="https://github.com/Zygo/bees.git"
+ inherit git-r3
+else
+ SRC_URI="https://github.com/Zygo/bees/archive/v${PV}.tar.gz -> ${P}.tar.gz"
+ KEYWORDS="~amd64"
+fi
+
+LICENSE="GPL-3"
+SLOT="0"
+IUSE="tools"
+
+DEPEND="
+ >=sys-apps/util-linux-2.30.2
+ >=sys-fs/btrfs-progs-4.1
+"
+RDEPEND="${DEPEND}"
+
+CONFIG_CHECK="~BTRFS_FS"
+ERROR_BTRFS_FS="CONFIG_BTRFS_FS: bees does currently only work with btrfs"
+
+PATCHES=(
+ "${FILESDIR}/6001-lib-fix-non-local-lambda-expression-cannot-have-a-ca.patch"
+ "${FILESDIR}/6002-context-workaround-to-prevent-LOGICAL_INO-and-btrfs-.patch"
+)
+
+pkg_pretend() {
+ if [[ ${MERGE_TYPE} != buildonly ]]; then
+ if kernel_is -lt 4 4 3; then
+ ewarn "Kernel versions below 4.4.3 lack critical features needed for bees to"
+ ewarn "properly operate, so it won't work. It's recommended to run at least"
+ ewarn "kernel version 4.11 for best performance and reliability."
+ ewarn
+ elif kernel_is -lt 4 11; then
+ ewarn "With kernel versions below 4.11, bees may severely degrade system performance"
+ ewarn "and responsiveness. Especially, the kernel may deadlock while bees is"
+ ewarn "running, it's recommended to run at least kernel 4.11."
+ ewarn
+ elif kernel_is -lt 4 14 29; then
+ ewarn "With kernel versions below 4.14.29, bees may generate a lot of bogus WARN_ON()"
+ ewarn "messages in the kernel log. These messages can be ignored and this is fixed"
+ ewarn "with more recent kernels:"
+ ewarn "# WARNING: CPU: 3 PID: 18172 at fs/btrfs/backref.c:1391 find_parent_nodes+0xc41/0x14e0"
+ ewarn
+ fi
+ if kernel_is -lt 5 3 4; then
+ ewarn "With kernel versions below 5.3.4, bees may trigger a btrfs bug when running"
+ ewarn "btrfs-balance in parallel. This may lead to meta-data corruption in the worst"
+ ewarn "case. Especially, kernels 5.1.21 and 5.2.21 should be avoided. Kernels 5.0.x"
+ ewarn "after 5.0.21 should be safe. In the best case, affected kernels may force"
+ ewarn "the device RO without writing corrupted meta-data. More details:"
+ ewarn "https://github.com/Zygo/bees/blob/master/docs/btrfs-kernel.md"
+ ewarn
+ fi
+ elog "Bees recommends running the latest current kernel for performance and"
+ elog "reliability reasons, see README.md."
+ fi
+}
+
+src_configure() {
+ cat >localconf <<-EOF || die
+ LIBEXEC_PREFIX=/usr/libexec
+ PREFIX=/usr
+ LIBDIR="$(get_libdir)"
+ SYSTEMD_SYSTEM_UNIT_DIR="$(systemd_get_systemunitdir)"
+ DEFAULT_MAKE_TARGET=all
+ EOF
+ if [[ ${PV} != "9999" ]] ; then
+ cat >>localconf <<-EOF || die
+ BEES_VERSION=v${PV}
+ EOF
+ fi
+ if use tools; then
+ echo OPTIONAL_INSTALL_TARGETS=install_tools >>localconf || die
+ fi
+}
diff --git a/sys-fs/bees/files/6001-lib-fix-non-local-lambda-expression-cannot-have-a-ca.patch b/sys-fs/bees/files/6001-lib-fix-non-local-lambda-expression-cannot-have-a-ca.patch
new file mode 100644
index 00000000000..60ab748eb2b
--- /dev/null
+++ b/sys-fs/bees/files/6001-lib-fix-non-local-lambda-expression-cannot-have-a-ca.patch
@@ -0,0 +1,38 @@
+From 566df54a3f7458559b75455a95b1991b515ba6bf Mon Sep 17 00:00:00 2001
+From: Zygo Blaxell <zblaxell@thirteen.furryterror.org>
+Date: Wed, 12 Jun 2019 21:27:50 -0400
+Subject: [PATCH 1/2] lib: fix non-local lambda expression cannot have a
+ capture-default
+
+We got away with this because GCC 4.8 (and apparently every GCC prior
+to 9) didn't notice or care, and because there is nothing referenced
+inside the lambda function body that isn't accessible from any other
+kind of function body (i.e. the capture wasn't needed at all).
+
+GCC 9 now enforces what the C++ standard said all along: there is
+no need to allow capture-default in this case, so it is not.
+
+Fix by removing the offending capture-default.
+
+Fixes: https://github.com/Zygo/bees/issues/112
+Signed-off-by: Zygo Blaxell <bees@furryterror.org>
+---
+ lib/error.cc | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/lib/error.cc b/lib/error.cc
+index f2a6db0..1d16a0a 100644
+--- a/lib/error.cc
++++ b/lib/error.cc
+@@ -32,7 +32,7 @@ namespace crucible {
+
+ // FIXME: could probably avoid some of these levels of indirection
+ static
+- function<void(string s)> current_catch_explainer = [&](string s) {
++ function<void(string s)> current_catch_explainer = [](string s) {
+ cerr << s << endl;
+ };
+
+--
+2.23.0
+
diff --git a/sys-fs/bees/files/6002-context-workaround-to-prevent-LOGICAL_INO-and-btrfs-.patch b/sys-fs/bees/files/6002-context-workaround-to-prevent-LOGICAL_INO-and-btrfs-.patch
new file mode 100644
index 00000000000..6d8d2ea530d
--- /dev/null
+++ b/sys-fs/bees/files/6002-context-workaround-to-prevent-LOGICAL_INO-and-btrfs-.patch
@@ -0,0 +1,95 @@
+From f731ea8016c570243c783adef96681b535d9c927 Mon Sep 17 00:00:00 2001
+From: Zygo Blaxell <bees@furryterror.org>
+Date: Tue, 19 Nov 2019 16:01:31 -0500
+Subject: [PATCH 2/2] context: workaround to prevent LOGICAL_INO and btrfs
+ balance from running concurrently
+
+This avoids some kernel bugs. One of them is fixed in 5.3.4 and later:
+
+ efad8a853a "Btrfs: fix use-after-free when using the tree modification log"
+
+There are apparently others in current kernels, so for now just put bees
+on pause until the balance is done.
+
+At some point we may want to provide an option to disable this
+workaround; however, running bees and balance at the same time makes
+neither particularly fast, so maybe we'll just leave it this way.
+
+Signed-off-by: Zygo Blaxell <bees@furryterror.org>
+---
+ src/bees-context.cc | 31 +++++++++++++++++++++++++++++++
+ src/bees.h | 4 ++++
+ 2 files changed, 35 insertions(+)
+
+diff --git a/src/bees-context.cc b/src/bees-context.cc
+index 4e0a43e..0665019 100644
+--- a/src/bees-context.cc
++++ b/src/bees-context.cc
+@@ -760,11 +760,42 @@ BeesResolveAddrResult::BeesResolveAddrResult()
+ {
+ }
+
++void
++BeesContext::wait_for_balance()
++{
++ Timer balance_timer;
++ BEESNOTE("WORKAROUND: waiting for balance to stop");
++ while (true) {
++ btrfs_ioctl_balance_args args;
++ memset_zero<btrfs_ioctl_balance_args>(&args);
++ const int ret = ioctl(root_fd(), BTRFS_IOC_BALANCE_PROGRESS, &args);
++ if (ret < 0) {
++ // Either can't get balance status or not running, exit either way
++ break;
++ }
++
++ if (!(args.state & BTRFS_BALANCE_STATE_RUNNING)) {
++ // Balance not running, doesn't matter if paused or cancelled
++ break;
++ }
++
++ BEESLOGDEBUG("WORKAROUND: Waiting " << balance_timer << "s for balance to stop");
++ sleep(BEES_BALANCE_POLL_INTERVAL);
++ }
++}
++
+ BeesResolveAddrResult
+ BeesContext::resolve_addr_uncached(BeesAddress addr)
+ {
+ THROW_CHECK1(invalid_argument, addr, !addr.is_magic());
+ THROW_CHECK0(invalid_argument, !!root_fd());
++
++ // Is there a bug where resolve and balance cause a crash (BUG_ON at fs/btrfs/ctree.c:1227)?
++ // Apparently yes, and more than one.
++ // Wait for the balance to finish before we run LOGICAL_INO
++ wait_for_balance();
++
++ // Time how long this takes
+ Timer resolve_timer;
+
+ // There is no performance benefit if we restrict the buffer size.
+diff --git a/src/bees.h b/src/bees.h
+index da87d88..5c9375c 100644
+--- a/src/bees.h
++++ b/src/bees.h
+@@ -114,6 +114,9 @@ const size_t BEES_TRANSID_FACTOR = 10;
+ // The actual limit in LOGICAL_INO seems to be 2730, but let's leave a little headroom
+ const size_t BEES_MAX_EXTENT_REF_COUNT = 2560;
+
++// Wait this long for a balance to stop
++const double BEES_BALANCE_POLL_INTERVAL = 60.0;
++
+ // Flags
+ const int FLAGS_OPEN_COMMON = O_NOFOLLOW | O_NONBLOCK | O_CLOEXEC | O_NOATIME | O_LARGEFILE | O_NOCTTY;
+ const int FLAGS_OPEN_DIR = FLAGS_OPEN_COMMON | O_RDONLY | O_DIRECTORY;
+@@ -708,6 +711,7 @@ class BeesContext : public enable_shared_from_this<BeesContext> {
+ void set_root_fd(Fd fd);
+
+ BeesResolveAddrResult resolve_addr_uncached(BeesAddress addr);
++ void wait_for_balance();
+
+ BeesFileRange scan_one_extent(const BeesFileRange &bfr, const Extent &e);
+ void rewrite_file_range(const BeesFileRange &bfr);
+--
+2.23.0
+
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/
@ 2020-04-17 15:10 Joonas Niilola
0 siblings, 0 replies; 5+ messages in thread
From: Joonas Niilola @ 2020-04-17 15:10 UTC (permalink / raw
To: gentoo-commits
commit: ca424c044ad8c11f2b0838508e1fa1d3276fdb72
Author: Kai Krakow <kai <AT> kaishome <DOT> de>
AuthorDate: Thu Apr 16 19:34:11 2020 +0000
Commit: Joonas Niilola <juippis <AT> gentoo <DOT> org>
CommitDate: Fri Apr 17 15:10:32 2020 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ca424c04
sys-fs/bees: Add musl hack to 0.6.1-r2
Package-Manager: Portage-2.3.89, Repoman-2.3.20
See-also: https://github.com/Zygo/bees/pull/143
Closes: https://bugs.gentoo.org/713960
Signed-off-by: Kai Krakow <kai <AT> kaishome.de>
Closes: https://github.com/gentoo/gentoo/pull/15374
Signed-off-by: Joonas Niilola <juippis <AT> gentoo.org>
sys-fs/bees/bees-0.6.1-r2.ebuild | 3 +-
...K-musl-does-not-define-pthread_getname_np.patch | 49 ++++++++++++++++++++++
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/sys-fs/bees/bees-0.6.1-r2.ebuild b/sys-fs/bees/bees-0.6.1-r2.ebuild
index 38c83939e49..3f4ce0f69aa 100644
--- a/sys-fs/bees/bees-0.6.1-r2.ebuild
+++ b/sys-fs/bees/bees-0.6.1-r2.ebuild
@@ -1,4 +1,4 @@
-# Copyright 1999-2019 Gentoo Authors
+# Copyright 2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
EAPI=7
@@ -33,6 +33,7 @@ PATCHES=(
"${FILESDIR}/6001-lib-fix-non-local-lambda-expression-cannot-have-a-ca.patch"
"${FILESDIR}/6002-context-workaround-to-prevent-LOGICAL_INO-and-btrfs-.patch"
"${FILESDIR}/6003-process-Fix-gettid-ambiguity-with-glibc-2.30.patch"
+ "${FILESDIR}/0001-HACK-musl-does-not-define-pthread_getname_np.patch"
)
pkg_pretend() {
diff --git a/sys-fs/bees/files/0001-HACK-musl-does-not-define-pthread_getname_np.patch b/sys-fs/bees/files/0001-HACK-musl-does-not-define-pthread_getname_np.patch
new file mode 100644
index 00000000000..5f542956d88
--- /dev/null
+++ b/sys-fs/bees/files/0001-HACK-musl-does-not-define-pthread_getname_np.patch
@@ -0,0 +1,49 @@
+From ba43aef5e264abd4b24baa1ca9f042e30e565155 Mon Sep 17 00:00:00 2001
+From: Kai Krakow <kai@kaishome.de>
+Date: Thu, 16 Apr 2020 21:21:57 +0200
+Subject: [PATCH] HACK: musl does not define pthread_getname_np()
+
+The glibc alternative musl does not define this setter function. This
+fix is probably wrong because it resets the thread names used for
+logging internally to empty strings.
+
+Link: https://www.openwall.com/lists/musl/2019/07/17/3
+---
+ lib/task.cc | 2 ++
+ src/bees.cc | 2 ++
+ 2 files changed, 4 insertions(+)
+
+diff --git a/lib/task.cc b/lib/task.cc
+index 4bdaeec..5b03335 100644
+--- a/lib/task.cc
++++ b/lib/task.cc
+@@ -104,7 +104,9 @@ namespace crucible {
+
+ char buf[24];
+ memset(buf, '\0', sizeof(buf));
++#ifdef _GNU_SOURCE
+ DIE_IF_MINUS_ERRNO(pthread_getname_np(pthread_self(), buf, sizeof(buf)));
++#endif
+ Cleanup pthread_name_cleaner([&]() {
+ pthread_setname_np(pthread_self(), buf);
+ });
+diff --git a/src/bees.cc b/src/bees.cc
+index aa55af0..c1cc8cb 100644
+--- a/src/bees.cc
++++ b/src/bees.cc
+@@ -160,10 +160,12 @@ BeesNote::get_name()
+ // OK try the pthread name next.
+ char buf[24];
+ memset(buf, '\0', sizeof(buf));
++#ifdef _GNU_SOURCE
+ int err = pthread_getname_np(pthread_self(), buf, sizeof(buf));
+ if (err) {
+ return string("pthread_getname_np: ") + strerror(err);
+ }
++#endif
+ buf[sizeof(buf) - 1] = '\0';
+
+ // thread_getname_np returns process name
+--
+2.24.1
+
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/
@ 2021-01-21 7:14 Joonas Niilola
0 siblings, 0 replies; 5+ messages in thread
From: Joonas Niilola @ 2021-01-21 7:14 UTC (permalink / raw
To: gentoo-commits
commit: 816bbe85535e4e3bfb07940b4529306a889babf4
Author: Kai Krakow <kai <AT> kaishome <DOT> de>
AuthorDate: Wed Jan 6 19:57:46 2021 +0000
Commit: Joonas Niilola <juippis <AT> gentoo <DOT> org>
CommitDate: Thu Jan 21 06:20:09 2021 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=816bbe85
sys-fs/bees: Rebased musl patch for -9999
Package-Manager: Portage-3.0.12, Repoman-3.0.2
Signed-off-by: Kai Krakow <kai <AT> kaishome.de>
Closes: https://github.com/gentoo/gentoo/pull/18968
Signed-off-by: Joonas Niilola <juippis <AT> gentoo.org>
sys-fs/bees/bees-9999.ebuild | 2 +-
...K-musl-does-not-define-pthread_getname_np.patch | 50 ++++++++++++++++++++++
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/sys-fs/bees/bees-9999.ebuild b/sys-fs/bees/bees-9999.ebuild
index 6c38f7b9c91..d656c4e6cde 100644
--- a/sys-fs/bees/bees-9999.ebuild
+++ b/sys-fs/bees/bees-9999.ebuild
@@ -29,7 +29,7 @@ RDEPEND="${DEPEND}"
CONFIG_CHECK="~BTRFS_FS"
ERROR_BTRFS_FS="CONFIG_BTRFS_FS: bees does currently only work with btrfs"
-PATCHES=( "${FILESDIR}/0001-HACK-musl-does-not-define-pthread_getname_np.patch" )
+PATCHES=( "${FILESDIR}/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch" )
pkg_pretend() {
if [[ ${MERGE_TYPE} != buildonly ]]; then
diff --git a/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch b/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch
new file mode 100644
index 00000000000..6b8224a5a56
--- /dev/null
+++ b/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch
@@ -0,0 +1,50 @@
+From 02bdb636be5abd15836e5e43e9b53517595686c9 Mon Sep 17 00:00:00 2001
+From: Kai Krakow <kai@kaishome.de>
+Date: Thu, 16 Apr 2020 21:21:57 +0200
+Subject: [PATCH v9999] HACK: musl does not define pthread_getname_np()
+
+The glibc alternative musl does not define this setter function. This
+fix is probably wrong because it resets the thread names used for
+logging internally to empty strings.
+
+Link: https://www.openwall.com/lists/musl/2019/07/17/3
+Signed-off-by: Kai Krakow <kai@kaishome.de>
+---
+ lib/task.cc | 2 ++
+ src/bees.cc | 2 ++
+ 2 files changed, 4 insertions(+)
+
+diff --git a/lib/task.cc b/lib/task.cc
+index 937a477..85da59b 100644
+--- a/lib/task.cc
++++ b/lib/task.cc
+@@ -194,7 +194,9 @@ namespace crucible {
+ lock.unlock();
+
+ char buf[24] = { 0 };
++#ifdef _GNU_SOURCE
+ DIE_IF_MINUS_ERRNO(pthread_getname_np(pthread_self(), buf, sizeof(buf)));
++#endif
+ DIE_IF_MINUS_ERRNO(pthread_setname_np(pthread_self(), m_title.c_str()));
+
+ sched_param param = { .sched_priority = 0 };
+diff --git a/src/bees.cc b/src/bees.cc
+index 03a626c..3db5d73 100644
+--- a/src/bees.cc
++++ b/src/bees.cc
+@@ -152,10 +152,12 @@ BeesNote::get_name()
+ // OK try the pthread name next.
+ char buf[24];
+ memset(buf, '\0', sizeof(buf));
++#ifdef _GNU_SOURCE
+ int err = pthread_getname_np(pthread_self(), buf, sizeof(buf));
+ if (err) {
+ return string("pthread_getname_np: ") + strerror(err);
+ }
++#endif
+ buf[sizeof(buf) - 1] = '\0';
+
+ // thread_getname_np returns process name
+--
+2.26.2
+
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/
@ 2021-07-07 18:47 Ionen Wolkens
0 siblings, 0 replies; 5+ messages in thread
From: Ionen Wolkens @ 2021-07-07 18:47 UTC (permalink / raw
To: gentoo-commits
commit: 5ad5474983c2e84bd466d03f24c5040c8c853e92
Author: Kai Krakow <kai <AT> kaishome <DOT> de>
AuthorDate: Sat Jun 19 08:42:50 2021 +0000
Commit: Ionen Wolkens <ionen <AT> gentoo <DOT> org>
CommitDate: Wed Jul 7 18:44:07 2021 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5ad54749
sys-fs/bees: Drop conflicting musl hack + small fixes in 9999
An upcoming musl update will add the missing function.
Also:
* Fix installation with eprefix
* Fix style
* Inherit toolchain-funcs (for tc-export)
Package-Manager: Portage-3.0.20, Repoman-3.0.2
Signed-off-by: Kai Krakow <kai <AT> kaishome.de>
Signed-off-by: Ionen Wolkens <ionen <AT> gentoo.org>
sys-fs/bees/bees-9999.ebuild | 15 +++----
...K-musl-does-not-define-pthread_getname_np.patch | 50 ----------------------
2 files changed, 6 insertions(+), 59 deletions(-)
diff --git a/sys-fs/bees/bees-9999.ebuild b/sys-fs/bees/bees-9999.ebuild
index d656c4e6cde..7777d0ea45f 100644
--- a/sys-fs/bees/bees-9999.ebuild
+++ b/sys-fs/bees/bees-9999.ebuild
@@ -3,7 +3,7 @@
EAPI=7
-inherit linux-info systemd
+inherit linux-info systemd toolchain-funcs
DESCRIPTION="Best-Effort Extent-Same, a btrfs dedup agent"
HOMEPAGE="https://github.com/Zygo/bees"
@@ -29,8 +29,6 @@ RDEPEND="${DEPEND}"
CONFIG_CHECK="~BTRFS_FS"
ERROR_BTRFS_FS="CONFIG_BTRFS_FS: bees does currently only work with btrfs"
-PATCHES=( "${FILESDIR}/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch" )
-
pkg_pretend() {
if [[ ${MERGE_TYPE} != buildonly ]]; then
if kernel_is -lt 4 11; then
@@ -74,17 +72,16 @@ src_prepare() {
}
src_configure() {
+ tc-export CC CXX AR
cat >localconf <<-EOF || die
- LIBEXEC_PREFIX=/usr/libexec
- PREFIX=/usr
- LIBDIR="$(get_libdir)"
+ LIBEXEC_PREFIX="${EPREFIX}/usr/libexec"
+ PREFIX="${EPREFIX}/usr"
+ LIBDIR="${EPREFIX}/$(get_libdir)"
SYSTEMD_SYSTEM_UNIT_DIR="$(systemd_get_systemunitdir)"
DEFAULT_MAKE_TARGET=all
EOF
if [[ ${PV} != "9999" ]] ; then
- cat >>localconf <<-EOF || die
- BEES_VERSION=v${PV}
- EOF
+ echo BEES_VERSION=v${PV} >>localconf || die
fi
if use tools; then
echo OPTIONAL_INSTALL_TARGETS=install_tools >>localconf || die
diff --git a/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch b/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch
deleted file mode 100644
index 6b8224a5a56..00000000000
--- a/sys-fs/bees/files/v9999-0001-HACK-musl-does-not-define-pthread_getname_np.patch
+++ /dev/null
@@ -1,50 +0,0 @@
-From 02bdb636be5abd15836e5e43e9b53517595686c9 Mon Sep 17 00:00:00 2001
-From: Kai Krakow <kai@kaishome.de>
-Date: Thu, 16 Apr 2020 21:21:57 +0200
-Subject: [PATCH v9999] HACK: musl does not define pthread_getname_np()
-
-The glibc alternative musl does not define this setter function. This
-fix is probably wrong because it resets the thread names used for
-logging internally to empty strings.
-
-Link: https://www.openwall.com/lists/musl/2019/07/17/3
-Signed-off-by: Kai Krakow <kai@kaishome.de>
----
- lib/task.cc | 2 ++
- src/bees.cc | 2 ++
- 2 files changed, 4 insertions(+)
-
-diff --git a/lib/task.cc b/lib/task.cc
-index 937a477..85da59b 100644
---- a/lib/task.cc
-+++ b/lib/task.cc
-@@ -194,7 +194,9 @@ namespace crucible {
- lock.unlock();
-
- char buf[24] = { 0 };
-+#ifdef _GNU_SOURCE
- DIE_IF_MINUS_ERRNO(pthread_getname_np(pthread_self(), buf, sizeof(buf)));
-+#endif
- DIE_IF_MINUS_ERRNO(pthread_setname_np(pthread_self(), m_title.c_str()));
-
- sched_param param = { .sched_priority = 0 };
-diff --git a/src/bees.cc b/src/bees.cc
-index 03a626c..3db5d73 100644
---- a/src/bees.cc
-+++ b/src/bees.cc
-@@ -152,10 +152,12 @@ BeesNote::get_name()
- // OK try the pthread name next.
- char buf[24];
- memset(buf, '\0', sizeof(buf));
-+#ifdef _GNU_SOURCE
- int err = pthread_getname_np(pthread_self(), buf, sizeof(buf));
- if (err) {
- return string("pthread_getname_np: ") + strerror(err);
- }
-+#endif
- buf[sizeof(buf) - 1] = '\0';
-
- // thread_getname_np returns process name
---
-2.26.2
-
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/
@ 2024-08-08 14:12 Joonas Niilola
0 siblings, 0 replies; 5+ messages in thread
From: Joonas Niilola @ 2024-08-08 14:12 UTC (permalink / raw
To: gentoo-commits
commit: 2e6314580e10fb5fd67c06c834ad99bab7686589
Author: Kai Krakow <kai <AT> kaishome <DOT> de>
AuthorDate: Wed Jul 31 09:49:48 2024 +0000
Commit: Joonas Niilola <juippis <AT> gentoo <DOT> org>
CommitDate: Thu Aug 8 14:12:42 2024 +0000
URL: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2e631458
sys-fs/bees: update patchset
This updates the patchset for better memory fragmentation behavior and
also adds a patch to reduce logging pressure if a lot of new files have
been added. The latter patch is pending upstream.
Signed-off-by: Kai Krakow <kai <AT> kaishome.de>
Signed-off-by: Joonas Niilola <juippis <AT> gentoo.org>
sys-fs/bees/bees-9999.ebuild | 6 +--
...ote-abandoned-toxic-match-to-debug-log-le.patch | 36 ++++++++++++++++++
...Work-around-kernel-memory-fragmentatio_v2.patch | 43 ++++++++++++++++++++++
3 files changed, 82 insertions(+), 3 deletions(-)
diff --git a/sys-fs/bees/bees-9999.ebuild b/sys-fs/bees/bees-9999.ebuild
index 1b164eb56b08..e1d5da90f548 100644
--- a/sys-fs/bees/bees-9999.ebuild
+++ b/sys-fs/bees/bees-9999.ebuild
@@ -1,4 +1,4 @@
-# Copyright 1999-2023 Gentoo Authors
+# Copyright 1999-2024 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
EAPI=8
@@ -18,7 +18,6 @@ fi
LICENSE="GPL-3"
SLOT="0"
-IUSE=""
DEPEND="
>=sys-apps/util-linux-2.30.2
@@ -30,7 +29,8 @@ CONFIG_CHECK="~BTRFS_FS"
ERROR_BTRFS_FS="CONFIG_BTRFS_FS: bees does currently only work with btrfs"
PATCHES=(
- "${FILESDIR}/0001-HACK-crucible-Work-around-kernel-memory-fragmentatio.patch"
+ "${FILESDIR}/0001-context-demote-abandoned-toxic-match-to-debug-log-le.patch"
+ "${FILESDIR}/0002-HACK-crucible-Work-around-kernel-memory-fragmentatio_v2.patch"
)
pkg_pretend() {
diff --git a/sys-fs/bees/files/0001-context-demote-abandoned-toxic-match-to-debug-log-le.patch b/sys-fs/bees/files/0001-context-demote-abandoned-toxic-match-to-debug-log-le.patch
new file mode 100644
index 000000000000..387ae7f1100c
--- /dev/null
+++ b/sys-fs/bees/files/0001-context-demote-abandoned-toxic-match-to-debug-log-le.patch
@@ -0,0 +1,36 @@
+From fad9230cea8a9970980c36b442b71b864e69ded3 Mon Sep 17 00:00:00 2001
+From: Kai Krakow <kai@kaishome.de>
+Date: Sun, 30 Jun 2024 16:27:20 +0200
+Subject: [PATCH 1/2] context: demote "abandoned toxic match" to debug log
+ level
+
+This log message creates a overwhelmingly lot of messages in the system
+journal, leading to write-back flushing storms under high activity. As
+it is a work-around message, it is probably only useful to developers,
+thus demote to debug level.
+
+This fixes latency spikes in desktop usage after adding a lot of new
+files, especially since systemd-journal starts to flush caches if it
+sees memory pressure.
+
+Signed-off-by: Kai Krakow <kai@kaishome.de>
+---
+ src/bees-context.cc | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/bees-context.cc b/src/bees-context.cc
+index 09ec152..2251783 100644
+--- a/src/bees-context.cc
++++ b/src/bees-context.cc
+@@ -471,7 +471,7 @@ BeesContext::scan_one_extent(const BeesFileRange &bfr, const Extent &e)
+
+ // Hash is toxic
+ if (found_addr.is_toxic()) {
+- BEESLOGWARN("WORKAROUND: abandoned toxic match for hash " << hash << " addr " << found_addr << " matching bbd " << bbd);
++ BEESLOGDEBUG("WORKAROUND: abandoned toxic match for hash " << hash << " addr " << found_addr << " matching bbd " << bbd);
+ // Don't push these back in because we'll never delete them.
+ // Extents may become non-toxic so give them a chance to expire.
+ // hash_table->push_front_hash_addr(hash, found_addr);
+--
+2.44.2
+
diff --git a/sys-fs/bees/files/0002-HACK-crucible-Work-around-kernel-memory-fragmentatio_v2.patch b/sys-fs/bees/files/0002-HACK-crucible-Work-around-kernel-memory-fragmentatio_v2.patch
new file mode 100644
index 000000000000..b62957c05291
--- /dev/null
+++ b/sys-fs/bees/files/0002-HACK-crucible-Work-around-kernel-memory-fragmentatio_v2.patch
@@ -0,0 +1,43 @@
+From 291574986ccfaff8b620f6baa185a567f2f0cfb5 Mon Sep 17 00:00:00 2001
+From: Kai Krakow <kai@kaishome.de>
+Date: Sun, 9 Jul 2023 12:24:40 +0200
+Subject: [PATCH 2/2] HACK: crucible: Work around kernel memory fragmentation
+
+According to @Zygo:
+
+> This will drastically reduce the number of reflinks bees can handle
+> to a single extent, but that size is beyond more than enough for most
+> filesystems. Making the buffer smaller may also reduce the size of
+> vmallocs which might be aggravating the kernel's memory manager.
+
+v2:
+
+According to @Zygo:
+
+> Lately I've been running with vm.swappiness=0 and
+> BEES_MAX_EXTENT_REF_COUNT set to 9999. With those settings there's no
+> swapping at all.
+
+Link: https://github.com/Zygo/bees/issues/260#issuecomment-1627586574
+Link: https://github.com/Zygo/bees/issues/260#issuecomment-2068287231
+Signed-off-by: Kai Krakow <kai@kaishome.de>
+---
+ src/bees.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/bees.h b/src/bees.h
+index 4dbc005..a3c3872 100644
+--- a/src/bees.h
++++ b/src/bees.h
+@@ -96,7 +96,7 @@ const double BEES_TOO_LONG = 5.0;
+ const double BEES_TOXIC_SYS_DURATION = 0.1;
+
+ // Maximum number of refs to a single extent
+-const size_t BEES_MAX_EXTENT_REF_COUNT = (16 * 1024 * 1024 / 24) - 1;
++const size_t BEES_MAX_EXTENT_REF_COUNT = (10000) - 1;
+
+ // How long between hash table histograms
+ const double BEES_HASH_TABLE_ANALYZE_INTERVAL = BEES_STATS_INTERVAL;
+--
+2.44.2
+
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-08-08 14:12 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-11-28 16:34 [gentoo-commits] repo/gentoo:master commit in: sys-fs/bees/, sys-fs/bees/files/ Joonas Niilola
-- strict thread matches above, loose matches on Subject: below --
2020-04-17 15:10 Joonas Niilola
2021-01-21 7:14 Joonas Niilola
2021-07-07 18:47 Ionen Wolkens
2024-08-08 14:12 Joonas Niilola
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox