From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 8C14915A7D9 for ; Mon, 13 Mar 2023 14:35:01 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 60F1DE07D1; Mon, 13 Mar 2023 14:34:55 +0000 (UTC) Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id E9ED2E0769 for ; Mon, 13 Mar 2023 14:34:54 +0000 (UTC) Received: by mail-ot1-x32c.google.com with SMTP id p13-20020a9d744d000000b0069438f0db7eso6782129otk.3 for ; Mon, 13 Mar 2023 07:34:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678718094; h=content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:autocrypt:openpgp:from:references:to:subject:from:to:cc :subject:date:message-id:reply-to; bh=sOyF4yEHeOB/P0LhK7qCj0JJ0kPYYPpby8ADPlwFEqE=; b=SQmh0fB5gksm3AD113gsPtH9uvdKwkpMuJ2UjyyFQSQ6Mxc7dc6F/WJLitWJO4aPuO Egdm9Hzk5rxTkT1mfbi9u4BFr+1LYJRm3nFguEmEdclGzYvFhBW/ZVHew7qm1JQtkKlF qE2mGqyImKKgD55N2c17bWMM7zXojFCc/pQ3KLOaSgDiDj0RKxEni+EenfHESG6V6VYs JiXITRkwLEHFZKrUwoYYATTWKMEkWfVimP9+R0Dx3daZH8Y3YTu63RZLlU6NLG5TtA85 qYNa4vSJlL++YQvwb+rvKpLITYONNnU1nOzW1B849z8G+BA/PIltSK3P1JKGZXBY++hh j47g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678718094; h=content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:autocrypt:openpgp:from:references:to:subject :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sOyF4yEHeOB/P0LhK7qCj0JJ0kPYYPpby8ADPlwFEqE=; b=rw24rMDmdS8M0uQPs8KCOftZQ0coq8zkn9nKE+Xx7z05T3oZG409IGsaZ89oKwthAF 450yu4RM0+M8BFT92j6IXjC9wN7CTPVHieS1Xt9i68K+272Ft/40Vvh9S2arGXcivEGE VT7BJTiXuD7p/3V93Ze7EEbnlOS/0yHZM39jtmYhsjCnCfquJtg9rqUtldzedyuE4yg9 +JG0MPba8hPXCXRR3oOoGmKn6Y/LB7DGxuEJAtWGDJ/+8H2r/SRmP34cFpNipTndjzgH t8MDs5LNJkfZB5ZRluYipvZ71iWIkPMmKxX5mmXWr/sj8uvJAXaAQMlgCanmDReni98s M6Ag== X-Gm-Message-State: AO0yUKUBOtMZP8YcRas+SDrkvRrfNkbZIt/mcHvSUSF7O1Zo1P2zcYGi GSeI37v2kUHeuwQ6j8tUMuThlvhAgkM= X-Google-Smtp-Source: AK7set9ssb6ynuV1FFx4bTCuhhSqeBhwIdt6tkd2QQTc2kroGfUZad9W19tdc6/rDGXDkR2qh9lXTw== X-Received: by 2002:a05:6830:349f:b0:694:4f92:e7d9 with SMTP id c31-20020a056830349f00b006944f92e7d9mr10544854otu.28.1678718093998; Mon, 13 Mar 2023 07:34:53 -0700 (PDT) Received: from [10.8.8.6] ([107.179.20.190]) by smtp.gmail.com with ESMTPSA id y191-20020a4a45c8000000b005250c840e64sm3160248ooa.3.2023.03.13.07.34.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Mar 2023 07:34:53 -0700 (PDT) Subject: Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card To: gentoo-user@lists.gentoo.org References: <57322874-e9c0-2f2c-8994-43438fe72995@gmail.com> From: Dale Openpgp: preference=signencrypt Autocrypt: addr=rdalek1967@gmail.com; prefer-encrypt=mutual; keydata= mQINBGFSciYBEADcEGMyJBSuavKO/XKUVvgkxck7Nl8Iuu8N2lcnRji/rSKg5c1Acix1ll9i oW8JBCHwvn0+Xy60BvEsqcup3YSHw5STl/bR1ePEehtnYrg8FdjdS91+B805RfnKMm69rFVI wLSBHQrSG1yxHd8CloWoEdhmVtP24buajbh114bgXd9ahtpZrCVMrWdWYUg2mEXguGV5uNAh Rf8SWxDNc79w24JxsV34a8niMUYMjzWr0rafIbzk732X38vGjVMLo/2mMpkbp9mPp++LHoY+ 0Pet8zxxdXPJSCd475kza1AD+hhSyBZXB9yknYWgyY3cZe1rGmooJSi2KX4QxO7npwLThcO1 be6KKRkd35+Fi/a1BzVOHsZMiK/gcwxEFoMd27gir4ehaeHJfFXl+65w4hj0EsOZSxrJrm2C R50g5By2czSKP1bADEygFNpIJj51AR+wM88NImG2RPtlT2maYBzazvF05g65cdHXGp1C7W5P wwwKU2DgABB2t7N7z5A69LnryBRw4zUYDRRYLTYlBlYgg+xILm2c0OrBdxJgLJa7JE50Eo25 d3PFwt9J0gYvqy6sPFLl9So0sDg9zm0hKQtXOP5kgropUFGrNoJI+mjwF4rYLRBVzZwNAvlO OhEvHubBo3mEllv4x+FeptwXZxlk7gUsdqI8AxnFB8K9wi6FVQARAQABtBtEYWxlIDxyZGFs ZWsxOTY3QGdtYWlsLmNvbT6JAk4EEwEIADgCGyMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AW IQQSG1h01ruv/WNXc3Q3RqOgiQH1GwUCYVJy8gAKCRA3RqOgiQH1G+waEACeTZCt77jnRAmQ AV7otKuZekDWiLi3Eig8tj5ZJiCNSYA/hIxzmexRP0GMqjitcXK1iGwWcvMzzvIq30GAjIfB 4BR38cnXbtBa6fNewiT7QaZe/Hn6yBRldXNQypzbHy+/o27bUEy+oX4rE7etUgEHQAjuw7xz XFWg4tH1/KJvsOVY5upnWc5LdxYhsuQ3dQD4b22GsK0pOBDfb9PiirYM8eGKvrVuq4E/c75z lDDFhINl18lNZ9D0ZFL3IkTjHsAAqFH9uhnnEB8CWdHbBewPEfRaOhBUYWZ3Q8uTkmDgZT8q D9jlvLEdw7Nh2ApdxoepnI/4D+ql2Gr4DtH7SEPydr5gcf1Qr/2bXRb1hAYnIVcbncs/Bm3Z bkRKPVWMfE3Fusa+p5hMzixk0YysMaTHlc7mYRYAEZGnPMXnmcCbetwARU7A0yz1M1kCMOAQ Lsz8KH5kv3cRenMB6SFfjND2JfAK61H5TtnPq3L8noS2ZykRYxq9Nm3X64O1tJojIKBoZFr8 AwYNCvqC6puUyGMuzHPh7jPof8glfrrEKIYUvNPGMDoVX3IGetxh/9l6NcxgFA4JGoR+LS3C zmeNrwlllAe3OEUfKoWVQ+pagpSdM+8hHolaSda4Ys66Z3fCR4ZvcTqfhTAVskpqdXa4isAk 7vTcXu3L499ttywEp7rJTbkCDQRhUnImARAAncUdVhmtRr59zqpTUppKroQYlzR0jv8oa7DG K4gakTAT2N7evnI9wpssmzyVk8VEiLzhnFQ/Ol3FRt6hZCXDJt0clyHOyTfvz/MNFttWuZTc mLpSvmRR6VRjAH+Tz3Eam2xUw3PGuH97BcXQ3NnX3msv1UDxtxxBu6e2YrdeOhrCUSgzokcJ 98ChUNy934cgepPybAI12lSWqVFQ1aG7jExZfiUk+333fPSDbpKoZbTW5YJLXbycmW/C1IWL qYQyNjRWKaGoJtUWFhhmNiOQct7n90aKivNVPavmN+UQ9LlMaINtf9T6XCzLfogCFsulDCDJ 0yNQLDTurHaB4E71xoctgXmLLq9z1RQ0W2XiVAAOZQj6K3+d0AOUjDhCQ2QW8dUSq0ckkZXV DKVJOGS8Nhf2eIWIqRnP3AcUiiaiFGqUaVUmUAZ6h/oJmgghEu/1S+pcuUKU5i69+XCZ3hH2 Jzwzbf7K+FAIkOhCfHncF8i1N1pk00pOVykNnqHTfFo3qFusHt0ZWgXVnnn4pYdXqZNoDhvF BRE5Vm4k/k96Pw8HRx6Os6eFSRrlqGzRgqsu86FekxusXB9UGv4lJhtU/J+8MRWsh22K718s DbQnABicGKFz1qQlWvcf59oTByhLINJCBt1WXl+TzJDXepr3QSkqmK41dO9Hob97C9dMiK8A EQEAAYkCNgQYAQgAIAIbDBYhBBIbWHTWu6/9Y1dzdDdGo6CJAfUbBQJhUnLyAAoJEDdGo6CJ AfUbVHIQAKSWw620vPhR3A/njU2z77F3z/Jk+HTKdE3fIyWSWdkYN7CBFL0NguOMP30WZ+qE sJhZu7T5hf251MwQUUt27xlfnKYOmQs7CqONlXuXlGZI6WufrUjxNcVz+5gJsqvUWuuJWsgg sDmE92IBnfG/f81fPHWQyfr/SF4wYDMyoFp5xCCQpp1zB63iuFvvrhxBkEHzmbRtVDOhl0Xp BVEDR1w3QRACw9QJD/KM05Czv9JNQYlwinWO/OaQ9cMlUpKLgswUPg9IZ5vucxScfuAUA5uC B1jlAQ8ZPlVukBmbEv5RGOv+lpuEbA3YDMVtEeH4YMFbjt/+vH3Cr2vTbp5JlpByLburJEH0 WXZLUawEfUsZvVwpOuJK75vaa2HYXee+Cb3iCIzwfIfctdlqzUcbGRczlRNM59hpvj4z29Gh 3kAxVHItAYq54ikxQ9l4hQ8s9sLYPbX/WtcBxNX8crBSw0FLnmzGleVEtBHyqtt5CLzQNgrj GYWl1vKDUmRPw1CdZ1c+fMN9CY11jOM5B5ZnqZWfDeVYO2iJ5SuvTycChexCb8WYn1bdCBIo bBtga2RBXbVt4Mh9E4owsszefn51MwfjXxB20Fc5k3GU1AVpTCMs3ayYCzo0b2pvEvdjtDcA CYLEFPWgaFX9iQAM/CDfKvTtvgGWpqtCL2raq/mQoJEU Message-ID: Date: Mon, 13 Mar 2023 09:34:52 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0 SeaMonkey/2.53.15 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Archives-Salt: 564ccb51-e9e9-44b9-b5a7-4864f9bd81ee X-Archives-Hash: 6c3949272d476c1cc7af49d534ffe789 Rich Freeman wrote: > On Mon, Mar 13, 2023 at 8:24 AM Dale wrote: >> According to my google searches, PCIe x4 is faster >> than PCIe x1. It's why some cards are PCIe x8 or x16. I think video >> cards are usually x16. My question is, given the PCIe x4 card costs >> more, is it that much faster than a PCIe x1? > It could be slower than PCIe x1, because you didn't specify the version. > > PCIe uses lanes. Each lane provides a certain amount of bandwidth > depending on the version in use. > > For example, a v1 4x card has 1 GB/s of bandwidth. A v4 1x card has > 2GB/s of bandwidth. > > Note that slot size is only loosely coupled with the number of lanes. > Lots of motherboards have a second 16x slot that only provides 4-8 > lanes to save on the cost of a PCIe swich. You can also use adapters > to connect a 16x card to a 1x slot, or you might find a motherboard > that has an open-ended slot so that you can just fit a 16x card onto > the 1x slot. It will of course only use a single lane that way. > > So what you need to do is consider the following: > > 1. How much bandwidth do you actually need? If you're using spinning > disks you aren't going to sustain more than 200MB/s to a single drive, > and the odds of having 10 drives using all that bandwidth are pretty > low. If you're using SSDs then you're more likely to max them out > since the seek cost is much lower. > 2. What PCIe version does your motherboard support? Sticking a v4 > card on an old motherboard that only supports v2 is going to result in > it running at v2 speeds, so don't pay a premium for something you > won't use. Likewise, if they cut down on the number of lanes assuming > they'll have more bandwidth you might have less than you expected to > have. > > Then look up the number of lanes and the PCIe version and see what you > can expect: > https://en.wikipedia.org/wiki/PCI_Express#History_and_revisions > > I think odds are you aren't going to want to pay a premium if you're > just using spinning disks. If you actually wanted solid state storage > then I'd also be avoiding SATA and trying to use NVMe, though doing > that at scale requires a lot of IO, and that will cost you quite a > bit. There is a reason your motherboard has mostly 1x slots - PCIe > lanes are expensive to support. On most consumer motherboards they're > only handled by the CPU, and consumer CPUs are very limited in how > many they offer. Higher end motherboards may have a switch and offer > more lanes, but they'll still bottleneck if they're all maxed out > getting into the CPU. If you buy a server CPU for several thousand > dollars one of the main features they offer is a LOT more PCIe lanes, > so you can load up on NVMes and have them running at v4-5. (Typical > NVMe uses a 4x M.2 slot, and of course you can have 16x cards offering > multiples of those.) > > The whole setup is pretty analogous to networking. If you have a > computer with 4 network ports you can bond them together and run them > to a switch that supports this with 4 cables, and get 4x the > bandwidth. However, you can also get a single connection to run at > higher speeds (1Gb, 2.5Gb, 10Gb, etc), and you can do both. PCIe > lanes are just like bonded network cables - they are just pairs of > signal wires that use differential signaling, just like twisted pairs > in an ethernet cable. Longer slots just add more of them. Everything > is packet switched, so if there are more lanes it just spreads the > packets across them. Higher versions mean higher speeds in each lane. > This is why I asked.  I didn't even think about the different PCIe versions available.  My mobo, had to go dig out the manual, says it is PCIe 2.0.  This is a Gigabyte 970A-UD3P motherboard.  Yes, I'm thinking about building a new rig.  Turn this into a NAS maybe.  Anyway, I'm assuming 2.0 isn't the slowest or fastest but as you point out, it'll be a bottleneck.  Everything has a bottleneck somewhere.  The difference in price isn't that large.  What I did was, I saw the PCIe x1 and bought it.  It supported Linux.  Later on I noticed the PCIe x4 and then wondered if I should upgrade to that.  Given the limits of my mobo and the fact I won't be maxing out the drives anyway, I don't see the need to upgrade.  Your info pretty much makes that clear.  I might, just might, see a small difference when using pvmove.  Maybe.  Given that it generally maxes out the drive as it is, even if it does go faster, it won't be much.  Add in that when I start a pvmove, I go nap and do other things anyway, I won't notice it.  Last pvmove took a little over 19 hours.  Even 20 or 30 minutes isn't much difference in the grand scheme of things.  Another question.  My rig is getting a bit aged.  I have a AMD FX-8350 8 core CPU running at 4GHz.  I also have 32GBs of memory.  I've read that Intel currently has the best bang for buck on CPUs nowadays.  I'm open to the idea of switching.  As far as speed goes, if I built a new rig that is using a reasonably cost CPU and memory, would I see any real improvements?  When I say 'reasonably cost', I usually find the fastest/newest then drop down a bit to get out of that 'brand new' price point.  Generally, the difference in price is quite large but the difference in speed isn't that much.  Also, I got hard drives, I don't spend much on video cards either since I don't game, except solitaire.  So, let's say a mobo, CPU and memory.  What price range would I need to look for?  Just a rough idea.  I figure the CPU will be a few hundred.  Memory may be half that.  Mobo will likely be close to $200 or so.  I'm thinking $500 to $700 or so.  Then comes case, video card and all that.  The CPU, memory and mobo is the ones I try to buy all at once from the same vendor, like Newegg or Tigerdirect.  Thoughts?  Am I close? Thanks for the info.  At least I know I'm good on drive speed.  For my use anyway.  Dale :-)  :-)