You can also pipe it through zstd on the fly, for data that compresses well you'll often see 1.5 to 3× the raw throughput, so a gigabit link can effectively move 165–330MB/s.
-T0 uses all cores. Bump the level above -6 for more compression, drop it for more speed, but if your CPU can't keep up, high levels will actually slow it down. Already compressed data won't see much benefit.
That used to require a crossover cable; I've done precisely that (with a crossover cable) back before https://en.wikipedia.org/wiki/Medium-dependent_interface#Aut... became widespread. With a straight-through cable, you'd be connecting the Transmit (TX) pin of one adapter to the TX pin of the other one, and the Receive (RX) pin of one to the RX pin of the other — and neither device would "hear" the messages the other one was sending. A crossover cable flipped those wires, so each device's TX pin was connected to the RX pin on the other end, and both devices could "hear" each other.
But with auto MDI-X, each device would notice "hey, I'm sending but not receiving anything," and would try flipping its Transmit and Receive functions around (transmitting on the RX pin and receiving on the TX pin). Since each device waited a random period before doing that, it was very unlikely (nigh-impossible) that they would both flip at the exact same moment. And if they did, the second interval would most likely not be identical either.
I'm simplifying a bit in the explanation above, but that's the broad strokes. And that's how my carefully-labeled crossover cables started gathering dust. (And then I realized "hey wait, I can just use these as normal cables now", and pulled them back out of storage and mixed them with my normal patch cables).
Interesting. I don't remember my crossover cables being marked that way; you had to read the dot-matrix printing along the side to know. Color would have been way more handy.
I added labels (just a piece of paper under some clear tape) saying "CROSSOVER" at both ends of the cables, so that if I ever plugged a crossover cable into a place where I needed a straight-through, the label would be right there next to the plug.
Well I have to add that was interesting too. Because I remember you had to hold the ends up next to each other and compare the order of the colored wires. Same order left-to-right, not cross-over. Anything else, cross-over.
"It's possible to just connect two computers together with Ethernet" - on the front page of hackernews. Incredible. Ethernet over USB4 interdomain protocol at least had some novelty, but this...
Back in the day you needed a modified cable for this with RX/TX swapped on one end but nowadays probably all nic chips have auto-detect so you can take a normal patch cable and do it.
I bought bunch of ready made 10 meter cables for lan party dunno 15 or 20 years ago... Used with switch... Grey with nice paper label at end with "Cross over"...
Haven't had any issues with them ever in regular use... So pointless even back then...
Oh, I had no idea you could perform such a "broadcast" ping on link-local addresses. Now, that is a nice trick that will save me some typing between computers.
mDNS / Avahi is also great if the machine advertises itself: just use machinename.local after plugging it.
Though the interface needs to be configured for link-local addresses. In my experience, NetworkManager now periodically resets the link if it can't get a DHCP server to respond. I understand the rationale, but it used to be simpler to plug into a headless machine and be confident that you could just ssh in.
I would assume so as well, I'm not sure if all host operating systems act the same though. But you could probably just wait a couple of seconds and the interfaces would have assigned themselves link-local addresses.
This remind myself the good old 8bit days of using a cross-over RS232 cable to send a file from one computer to another. even at 30bps, it was much more reliable than write my data to a cassette tape on one computer and then reading on the other.
I have plugged two ZX Spectrums together and done LOAD "" on one and SAVE "THING" on the other to transfer data.
If you get away from BASIC commands and copy the tape routines up into RAM you can mess with the timing values, and with care you can get it from an average of 1800bps up to around IIRC 12000bps before things really fall apart. Probably with better interfacing you'd get even faster.
With more conservative values, this is how "speedloaders" worked. In the late 80s or early 90s one publisher released games on CD where it would load in the loader at normal rate then ramp up to some unholy speed that CD had the bandwidth and stability for but tape did not.
tbstream looks fun, but hard to use for the casual user -- not sure if it'll get simpler but I don't think I'll remember messing in configfs just to transfer a file:
The USB/Thunderbolt one is a nice catch and good to know if you want to move you data to a new PC. Dell XPS (with soldered SSD) to Framework move just took a few minutes by dd'ing through the created network interfaces (it's different from the tbstream). The limiting factor was the Dell's SSD actually.
It would be nice if this article defined a "ethernet patch cable". I think he's just using "patch" as a slang term for a short cable and it actually works with any length of standard cable, but I'm not certain.
Traditionally (pre-2000?), one had to use a special "crossover cable" to do direct connections like this, but apparently modern Gigabit ethernet adapters are able to detect this situation automatically?
https://en.wikipedia.org/wiki/Medium-dependent_interface#Aut... is what allowed the switchover to happen automatically, and it got folded into the 1000BASE-T standard. So yes, crossover cables are no longer required because the two Ethernet adapters at either end can negotiate between them to have one of them "flip" the meaning of its Receive (RX) and Transmit (TX) pins.
They are all just twisted pair cables, probably Cat5e or Cat6 etc. The solid ones are usually used for structured cabling, inside walls and trunking, that doesn't often move. Structured cabling is terminated into keystones or patch panels, the "female" end.
Patch cables are outside of walls and designed to be moved around and plugged and replugged. They are terminated with 8p8c "male" connectors, commonly called rj45. Stranded cable is used because it's much more flexible.
No, traditionally a patch cable is a cable that you would use in a networking rack to connect (to "patch") from say a network switch to the desk outlets or to another switch. A patch cable would explicitly be NOT a crossover.
The main difference between patch and "normal" is that the former is using stranded conductors and the latter solid conductors.
That makes the patch cable more flexible (physically)
Actually, switch to switch would be a crossover cable (pre auto MDI-X, of course). The rule was a crossover would connect devices of the same type, e.g. computer to computer, hub to hub, switch to switch; and patch for devices of different type, like switch to computer. So if you wanted to just plug a hub into the wall you would use a crossover cable, for example.
I am not sure that I know of any Ethernet (IEEE 802.3) standard cabling which uses solid-core wires rather than twisted-pair stranded wires. I can understand and see that many people consider "structured cabling" to consist of the former, but this article does not seem to mandate that construction, and indeed includes many types of twisted-pair and "patch" cables in the list of acceptable "structured cabling solutions."
Yeah, it's not patch vs structured. Patch cables are part of structured cabling. But you won't generally find solid core wires outside of structured cabling, because they are inflexible, unwieldly and in many cases can't even be terminated with RJ45 plugs anyway (they are often too thick).
To regular people it's probably enough just to say Ethernet cable. Outside of structured cabling you don't really need to care and probably don't have a choice anyway. If someone says patch cable they probably know a bit about structured cabling and couldn't help themselves.
Within structured cabling the rabbit hole is quite deep. Even within a single standard like Cat6 there are many possible cable constructions with different types of shielding and wire gauge etc. It becomes important when you need the network to certify as Cat6 and support PoE in a high noise industrial environment, for example. And, of course, Ethernet can run over other cable types anyway, like fibre.
I have a feeling this is some re-invention of the definitions happening.
It's just a holdover from the telephone networks, where you used a short cable to actually phsyically connect - patch[0] - two lines. Therefore - patch-panel and patch-cord or patch-cable.
For the time spent finding the ethernet adapter, plugging the cable, configuring IP addresses, and testing with pings, any decent portable SSD drives should have finished copying tens of GB of data.
For anyone today who's got copper wire stretched between any buildings, you are in danger and your configuration is quite unwise.
For every standalone building that has a ground, that building also has an electrical potential. This potential is not necessarily the same between two neighboring buildings. When you stretch a copper line between them, you are tempting fate, electrically, not to mention the dangers of lightning strikes and the elements attacking that connection.
It is generally recognized that fiber optics are not only very efficient and durable, but also impervious to electrical troubles of the type that may plague inter-building connections. Simply grab switches or routers that support at least one fiber-optic connection, and link up!
I did work for a small community college back in the 90s that had expanded their building multiple times since the 70s. The electrical potential between different parts of the building was significant enough to make wired Ethernet unreliable. (I never did understand why this was-- multiple electric service entrances with separate grounds, I assume.)
Because of this they were an early adopter of fiber-based networking. I remember being wowed by ATM over fiber running between closets at a blazing 155Mbps (with tons of 10BASE-T clients connected to FORE chassis switches).
Besides you worry too much. Entire cities were wired with cat5 hanged off steel cable between 9 to 15 story buildings in early 2000s, before fiber. Nothing burned down.
It seems a strange choice, for the demo and for the script, to manually configure addresses on both ends. If your TCP/IP stack is functioning properly, this will not be necessary. Once DHCP fails, you should get a pair of 169.254.0.0/16 (APIPA) addresses, and then Bob's your uncle.
Configuring all this manually adds extra complexity when it seems that the goal is simply to connect up your cable and let 'er fly.
Back in my college days, 169.254.x.y addresses were the bane of my existence, because my job was getting students' computers set up with the brand-new Ethernet connection in their dorm room. This usually required a tech to come out to their room and configure Windows 95 properly, with IP and DHCP instead of Novell Netware and IPX (I don't remember why Netware/IPX was the default on so many Windows 95 installs, but that was definitely what I found most often when I went to look at someone's computer that "wasn't connecting to the Internet".)
Occasionally we had someone who knew what they were doing and had their Windows 95 settings set up correctly, and then all we had to do was activate their room's Ethernet port and add their MAC address to the DHCP server's list of authorized MACs. But most often I or one of the other techs had to head down to the person's dorm room and set up their computer's network settings correctly before it would work. Once the 169.254 address was replaced by 192.168, my job was finally done. But there were times when that took some doing.
And if you prefer ipv6 you can as well just ping -6 ff02::1%eth0 (or eno1 or whatever your devices name in the particular subnet is) and receive all fe80 addresses of other hosts in the subnet.
Also, why is he talking about "ethernet"? Its the IP layer, not the ethernet layer...
Receiver: socat -u TCP6-LISTEN:1234,reuseaddr STDOUT | zstd -d -c | tar -xpf - -C /destination
Sender: tar -C /source -cf - directory | zstd -T0 -6 -c | socat -u STDIN 'TCP6:[fd42:dead:beef::2]:1234'
-T0 uses all cores. Bump the level above -6 for more compression, drop it for more speed, but if your CPU can't keep up, high levels will actually slow it down. Already compressed data won't see much benefit.
But with auto MDI-X, each device would notice "hey, I'm sending but not receiving anything," and would try flipping its Transmit and Receive functions around (transmitting on the RX pin and receiving on the TX pin). Since each device waited a random period before doing that, it was very unlikely (nigh-impossible) that they would both flip at the exact same moment. And if they did, the second interval would most likely not be identical either.
I'm simplifying a bit in the explanation above, but that's the broad strokes. And that's how my carefully-labeled crossover cables started gathering dust. (And then I realized "hey wait, I can just use these as normal cables now", and pulled them back out of storage and mixed them with my normal patch cables).
/looks longingly out the window, and catch my aging face in the reflection
Look at you with your fancy red Cat5 ;-)
Haven't had any issues with them ever in regular use... So pointless even back then...
mDNS / Avahi is also great if the machine advertises itself: just use machinename.local after plugging it.
Though the interface needs to be configured for link-local addresses. In my experience, NetworkManager now periodically resets the link if it can't get a DHCP server to respond. I understand the rationale, but it used to be simpler to plug into a headless machine and be confident that you could just ssh in.
If you get away from BASIC commands and copy the tape routines up into RAM you can mess with the timing values, and with care you can get it from an average of 1800bps up to around IIRC 12000bps before things really fall apart. Probably with better interfacing you'd get even faster.
With more conservative values, this is how "speedloaders" worked. In the late 80s or early 90s one publisher released games on CD where it would load in the loader at normal rate then ramp up to some unholy speed that CD had the bandwidth and stability for but tape did not.
"Computer to computer should not be seen without a device in between (unless playing ROTT with your housemates)"
35 years ago maybe :) I had serial and parallel socket, and routerless networking happening on my home systems :)
Use cases? Emergencies and maybe dedicated backup.
https://git.kernel.org/pub/scm/linux/kernel/git/westeri/thun...
Traditionally (pre-2000?), one had to use a special "crossover cable" to do direct connections like this, but apparently modern Gigabit ethernet adapters are able to detect this situation automatically?
Patch cables are outside of walls and designed to be moved around and plugged and replugged. They are terminated with 8p8c "male" connectors, commonly called rj45. Stranded cable is used because it's much more flexible.
edit: as for length, it depends. you won't get a gigabit over 150m of crap cat3, but you will get a link, even if it's 10 half-duplex.
The main difference between patch and "normal" is that the former is using stranded conductors and the latter solid conductors.
That makes the patch cable more flexible (physically)
Also probably a more flexible jacket and thinner. Vs fire resistant / plénum cables for fixed / in wall stuff.
And some had a switch to select the mode of the uplink port.
https://en.wikipedia.org/wiki/Structured_cabling
So I believe that, in the realm of Ethernet in particular, "patch" vs. "structured" is not a useful distinction.
To regular people it's probably enough just to say Ethernet cable. Outside of structured cabling you don't really need to care and probably don't have a choice anyway. If someone says patch cable they probably know a bit about structured cabling and couldn't help themselves.
Within structured cabling the rabbit hole is quite deep. Even within a single standard like Cat6 there are many possible cable constructions with different types of shielding and wire gauge etc. It becomes important when you need the network to certify as Cat6 and support PoE in a high noise industrial environment, for example. And, of course, Ethernet can run over other cable types anyway, like fibre.
It's just a holdover from the telephone networks, where you used a short cable to actually phsyically connect - patch[0] - two lines. Therefore - patch-panel and patch-cord or patch-cable.
https://www.merriam-webster.com/dictionary/patch
> Removable storage is quite slow unless you are willing to spend a lot of money.
An SSD good enough to copy tens of GB quickly is not cheap.
Not Warcraft though - some ancient DOS game.
For every standalone building that has a ground, that building also has an electrical potential. This potential is not necessarily the same between two neighboring buildings. When you stretch a copper line between them, you are tempting fate, electrically, not to mention the dangers of lightning strikes and the elements attacking that connection.
It is generally recognized that fiber optics are not only very efficient and durable, but also impervious to electrical troubles of the type that may plague inter-building connections. Simply grab switches or routers that support at least one fiber-optic connection, and link up!
Because of this they were an early adopter of fiber-based networking. I remember being wowed by ATM over fiber running between closets at a blazing 155Mbps (with tons of 10BASE-T clients connected to FORE chassis switches).
Besides you worry too much. Entire cities were wired with cat5 hanged off steel cable between 9 to 15 story buildings in early 2000s, before fiber. Nothing burned down.
Configuring all this manually adds extra complexity when it seems that the goal is simply to connect up your cable and let 'er fly.
Occasionally we had someone who knew what they were doing and had their Windows 95 settings set up correctly, and then all we had to do was activate their room's Ethernet port and add their MAC address to the DHCP server's list of authorized MACs. But most often I or one of the other techs had to head down to the person's dorm room and set up their computer's network settings correctly before it would work. Once the 169.254 address was replaced by 192.168, my job was finally done. But there were times when that took some doing.
Also, why is he talking about "ethernet"? Its the IP layer, not the ethernet layer...