From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 8A55315802F for ; Mon, 13 Mar 2023 12:48:31 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 3C571E07EF; Mon, 13 Mar 2023 12:48:26 +0000 (UTC) Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id C54BFE07D0 for ; Mon, 13 Mar 2023 12:48:25 +0000 (UTC) Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-53916ab0c6bso239835507b3.7 for ; Mon, 13 Mar 2023 05:48:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678711705; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hDIAPWC8L5VXrL/iurpGtQEqNWvOaMqYRDo1heXy6LM=; b=vwLVZNkaHNWymrmX+pK7fb/aIzn0GTi5Ukvn3D+9LPWGJ0+FUA0HKWkrYcDaFy0FEP 0OM3rw88QTbwnoOeoe4tN0SoRz404O3NsqsfTytnDCSu0eIqjuC9lXRgmarniBedTnVQ iBCN7XbxTKlpCcxctQS4TOfWr81JwWtTI0/4QYxc/JpJHdDgqSpo9RIJf7Gfm+v110fy YRVPockCaiktF/Ky1KX6AUs5O2yXPtlUG0XXKT99THP2RoNbbYN5BiwrAYUkavqCsQ0q vy60hXYj1n7hnCroBNGuUX1efroIffkhKQERq8IzEfuBaTB5qfYG1qQhNPQ+/F056DCu 2G/Q== X-Gm-Message-State: AO0yUKVZbSYnKY82onyQwUmE1PLfod9l2NhLEOpfnkTFY9EktEepebw/ QRrfYTUIT0lniUTcltAFXy8tSY61oHODOST2+CgpM3D+U5Q= X-Google-Smtp-Source: AK7set9XticCNFejV4NDurzhZFCzytPB3+NvmY8GDfjSOTCi7PK2zZS9j1/JiTV8Z7LpELcIAArFCd3gvJH6BGSCcJs= X-Received: by 2002:a81:4005:0:b0:532:e887:2c23 with SMTP id l5-20020a814005000000b00532e8872c23mr23357204ywn.9.1678711704660; Mon, 13 Mar 2023 05:48:24 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <57322874-e9c0-2f2c-8994-43438fe72995@gmail.com> In-Reply-To: <57322874-e9c0-2f2c-8994-43438fe72995@gmail.com> From: Rich Freeman Date: Mon, 13 Mar 2023 08:48:14 -0400 Message-ID: Subject: Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 589d97b6-7fae-4c40-ac97-7ca5090b68a1 X-Archives-Hash: 9cb2fca72638b14808fdade576abafbb On Mon, Mar 13, 2023 at 8:24=E2=80=AFAM Dale wrote: > > According to my google searches, PCIe x4 is faster > than PCIe x1. It's why some cards are PCIe x8 or x16. I think video > cards are usually x16. My question is, given the PCIe x4 card costs > more, is it that much faster than a PCIe x1? It could be slower than PCIe x1, because you didn't specify the version. PCIe uses lanes. Each lane provides a certain amount of bandwidth depending on the version in use. For example, a v1 4x card has 1 GB/s of bandwidth. A v4 1x card has 2GB/s of bandwidth. Note that slot size is only loosely coupled with the number of lanes. Lots of motherboards have a second 16x slot that only provides 4-8 lanes to save on the cost of a PCIe swich. You can also use adapters to connect a 16x card to a 1x slot, or you might find a motherboard that has an open-ended slot so that you can just fit a 16x card onto the 1x slot. It will of course only use a single lane that way. So what you need to do is consider the following: 1. How much bandwidth do you actually need? If you're using spinning disks you aren't going to sustain more than 200MB/s to a single drive, and the odds of having 10 drives using all that bandwidth are pretty low. If you're using SSDs then you're more likely to max them out since the seek cost is much lower. 2. What PCIe version does your motherboard support? Sticking a v4 card on an old motherboard that only supports v2 is going to result in it running at v2 speeds, so don't pay a premium for something you won't use. Likewise, if they cut down on the number of lanes assuming they'll have more bandwidth you might have less than you expected to have. Then look up the number of lanes and the PCIe version and see what you can expect: https://en.wikipedia.org/wiki/PCI_Express#History_and_revisions I think odds are you aren't going to want to pay a premium if you're just using spinning disks. If you actually wanted solid state storage then I'd also be avoiding SATA and trying to use NVMe, though doing that at scale requires a lot of IO, and that will cost you quite a bit. There is a reason your motherboard has mostly 1x slots - PCIe lanes are expensive to support. On most consumer motherboards they're only handled by the CPU, and consumer CPUs are very limited in how many they offer. Higher end motherboards may have a switch and offer more lanes, but they'll still bottleneck if they're all maxed out getting into the CPU. If you buy a server CPU for several thousand dollars one of the main features they offer is a LOT more PCIe lanes, so you can load up on NVMes and have them running at v4-5. (Typical NVMe uses a 4x M.2 slot, and of course you can have 16x cards offering multiples of those.) The whole setup is pretty analogous to networking. If you have a computer with 4 network ports you can bond them together and run them to a switch that supports this with 4 cables, and get 4x the bandwidth. However, you can also get a single connection to run at higher speeds (1Gb, 2.5Gb, 10Gb, etc), and you can do both. PCIe lanes are just like bonded network cables - they are just pairs of signal wires that use differential signaling, just like twisted pairs in an ethernet cable. Longer slots just add more of them. Everything is packet switched, so if there are more lanes it just spreads the packets across them. Higher versions mean higher speeds in each lane. --=20 Rich