From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id E3C7B1382C5 for ; Thu, 3 Dec 2020 11:36:07 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 4FDCAE0882; Thu, 3 Dec 2020 11:36:06 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 226A6E0882 for ; Thu, 3 Dec 2020 11:36:06 +0000 (UTC) Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 9601B341349 for ; Thu, 3 Dec 2020 11:36:04 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id 2E3D0481 for ; Thu, 3 Dec 2020 11:36:02 +0000 (UTC) From: "Andrew Ammerlaan" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Andrew Ammerlaan" Message-ID: <1606894475.c54f1d5636d1ca49e21634a81af3bf7a9609581d.andrewammerlaan@gentoo> Subject: [gentoo-commits] repo/proj/guru:master commit in: sys-fs/dwarfs/ X-VCS-Repository: repo/proj/guru X-VCS-Files: sys-fs/dwarfs/Manifest sys-fs/dwarfs/dwarfs-0.2.3.ebuild sys-fs/dwarfs/metadata.xml X-VCS-Directories: sys-fs/dwarfs/ X-VCS-Committer: andrewammerlaan X-VCS-Committer-Name: Andrew Ammerlaan X-VCS-Revision: c54f1d5636d1ca49e21634a81af3bf7a9609581d X-VCS-Branch: master Date: Thu, 3 Dec 2020 11:36:02 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: de5c507d-318c-4d93-800a-81a228b8ee21 X-Archives-Hash: fb9021586a0695d7ebf4334d2285d0c3 commit: c54f1d5636d1ca49e21634a81af3bf7a9609581d Author: Denis Reva gmail com> AuthorDate: Wed Dec 2 07:34:35 2020 +0000 Commit: Andrew Ammerlaan riseup net> CommitDate: Wed Dec 2 07:34:35 2020 +0000 URL: https://gitweb.gentoo.org/repo/proj/guru.git/commit/?id=c54f1d56 sys-fs/dwarfs: A fast high compression read-only file system I has just ported new package: https://github.com/mhx/dwarfs It's file-system like squashfs but compresses stronger I had to use built-in libs because they hard-coded, and furthermore they used partially So It no more require dev-cpp/folly, and I thought about deleting its. Package-Manager: Portage-3.0.9, Repoman-3.0.2 Signed-off-by: Denis Reva gmail.com> sys-fs/dwarfs/Manifest | 3 ++ sys-fs/dwarfs/dwarfs-0.2.3.ebuild | 93 +++++++++++++++++++++++++++++++++++++++ sys-fs/dwarfs/metadata.xml | 19 ++++++++ 3 files changed, 115 insertions(+) diff --git a/sys-fs/dwarfs/Manifest b/sys-fs/dwarfs/Manifest new file mode 100644 index 00000000..51b331f4 --- /dev/null +++ b/sys-fs/dwarfs/Manifest @@ -0,0 +1,3 @@ +DIST dwarfs-0.2.3.tar.gz 131021 BLAKE2B 4bdcbc7c775b4e20dc523f77e7fb84a9cd978df8c63bfd2300343bbf4dd49394e8f0fb1779ac990174c38acb2dbe1444404c518c22382da8e36e13e65343c568 SHA512 4638f3a9d712ead6701f3b739d0239ddf5f742c02cf4723079df702549c3af179b798b833bd69b77eeffb68df670fbcedff70ef8d7a88e4c9e5dd42b090eed63 +DIST fbthrift-2020.11.30.00.tar.gz 5274706 BLAKE2B ad13eb7b66f804c7a667746d72298e05726d1f78a49523a10f2712cc4dbea71999590a2c420e8081d32dc24a7d203e11d5c7e5c15f2627594295cd5533096dd9 SHA512 33b4c1a4cfc8783f99d5da8ae113f877e1f23586f5a854833107805fe35834604e677d7abe690cefeb70baed20cb3cc14449cfae35a9af89594974377bbe0329 +DIST folly-2020.11.30.00.tar.gz 3261437 BLAKE2B 5070819f1f8bac2030e761a8d2330379e8e5ba242d21330b98dcd598cb18e12046a9ad3fb6462f82dbc47c53cf0905c330e707201bfbbedb81f203ef1b30731e SHA512 e7e06bd8c8e38af345d20a04886b4497c4f0c9877263dbcb5a3f2d17c7a3252a2529086a5a04d07c2d3a457ca3930c43672fc4f74b7bdbc2ec0d07abdf1ed425 diff --git a/sys-fs/dwarfs/dwarfs-0.2.3.ebuild b/sys-fs/dwarfs/dwarfs-0.2.3.ebuild new file mode 100644 index 00000000..c5976474 --- /dev/null +++ b/sys-fs/dwarfs/dwarfs-0.2.3.ebuild @@ -0,0 +1,93 @@ +# Copyright 1999-2020 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=7 + +inherit cmake check-reqs + +DESCRIPTION="A fast very high compression read-only FUSE file system" +HOMEPAGE="https://github.com/mhx/dwarfs" + +FBTHRIFT_MAGIC="2020.11.30.00" +FOLLY_MAGIC="2020.11.30.00" + +SRC_URI="https://github.com/mhx/dwarfs/archive/v${PV}.tar.gz -> ${P}.tar.gz + https://github.com/facebook/fbthrift/archive/v${FBTHRIFT_MAGIC}.tar.gz -> fbthrift-${FBTHRIFT_MAGIC}.tar.gz + https://github.com/facebook/folly/archive/v${FOLLY_MAGIC}.tar.gz -> folly-${FOLLY_MAGIC}.tar.gz" + +LICENSE="GPL-3" +SLOT="0" +KEYWORDS="~amd64 ~x86" +IUSE="jemalloc" + +DEPEND="app-arch/zstd + app-arch/lz4 + app-arch/xz-utils + app-arch/snappy + dev-libs/boost[context,threads] + dev-libs/double-conversion + dev-libs/libfmt + dev-libs/libevent + dev-cpp/gflags + dev-cpp/glog[gflags] + dev-cpp/sparsehash + sys-libs/binutils-libs + sys-libs/libunwind + sys-fs/fuse:3 + sys-devel/flex + sys-devel/binutils:* + sys-libs/zlib + jemalloc? ( dev-libs/jemalloc )" +RDEPEND="dev-libs/boost[context,threads] + dev-libs/double-conversion + app-arch/zstd + app-arch/lz4 + app-arch/xz-utils + app-arch/snappy + sys-fs/fuse:3 + sys-libs/binutils-libs + sys-libs/libunwind" +BDEPEND="app-text/ronn + dev-util/cmake + sys-apps/sed + sys-devel/bison + virtual/pkgconfig" + +CHECKREQS_DISK_BUILD="512M" + +DOCS=( "README.md" "CHANGES.md" "TODO" ) + +src_unpack(){ + default + rm -d "${S}"/fbthrift/ + rm -d "${S}"/folly/ + mv "${WORKDIR}/fbthrift-${FBTHRIFT_MAGIC}" "${S}"/fbthrift/ + mv "${WORKDIR}/folly-${FBTHRIFT_MAGIC}" "${S}"/folly/ +} + +src_prepare(){ + einfo "setting path to $(get_libdir)" + pushd "${S}"/folly + sed "s/lib CACHE/$(get_libdir) CACHE/" -i CMakeLists.txt || die + sed "s/lib\/cmake\/folly CACHE/$(get_libdir)\/cmake\/folly CACHE/" -i CMakeLists.txt || die + popd + pushd "${S}"/fbthrift + sed "s/lib CACHE/$(get_libdir) CACHE/" -i CMakeLists.txt || die + sed "s/lib\/cmake\/fbthrift CACHE/$(get_libdir)\/cmake\/fbthrift CACHE/" -i CMakeLists.txt || die + popd + sed "s/DESTINATION lib/DESTINATION $(get_libdir)/" -i CMakeLists.txt || die + cmake_src_prepare +} + +pkg_postinst(){ + elog "Test shows that dwarfs compiled with Clang is substantially faster than GCC ones" + elog "See ${HOMEPAGE}" + elog "So you may want to compile it independently with Clang by the" + elog "https://wiki.gentoo.org/wiki/Clang" + elog "And with the per-package settings:" + elog "https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/Advanced#Per-package_environment_variables" + elog "Also you may find more information in the" + elog "${HOMEPAGE}" + elog "About creating: ${HOMEPAGE}/blob/main/doc/mkdwarfs.md" + elog "About mounting: ${HOMEPAGE}/blob/main/doc/dwarfs.md" +} diff --git a/sys-fs/dwarfs/metadata.xml b/sys-fs/dwarfs/metadata.xml new file mode 100644 index 00000000..17e2efa0 --- /dev/null +++ b/sys-fs/dwarfs/metadata.xml @@ -0,0 +1,19 @@ + + + + + denis7774@gmail.com + Denis Reva + Partial maintainer + + + DwarFS is a read-only file system with a focus on achieving very high compression ratios in particular for very redundant data. + This probably doesn't sound very exciting, because if it's redundant, it should compress well. However, I found that other read-only, compressed file systems don't do a very good job at making use of this redundancy. See here for a comparison with other compressed file systems. + DwarFS also doesn't compromise on speed and for my use cases I've found it to be on par with or perform better than SquashFS. For my primary use case, DwarFS compression is an order of magnitude better than SquashFS compression, it's 4 times faster to build the file system, it's typically faster to access files on DwarFS and it uses less CPU resources. + Distinct features of DwarFS are: + * Clustering of files by similarity using a similarity hash function. This makes it easier to exploit the redundancy across file boundaries. + * Segmentation analysis across file system blocks in order to reduce the size of the uncompressed file system. This saves memory when using the compressed file system and thus potentially allows for higher cache hit rates as more data can be kept in the cache. + * Highly multi-threaded implementation. Both the file system creation tool as well as the FUSE driver are able to make good use of the many cores of your system. + * Optional experimental Lua support to provide custom filtering and ordering functionality. + +