I don't really understand why the original ZMODEM spec did not specify these with charts in the first place (like ASCII charts):

https://github.com/jarkkojs/zmodem2/issues/7

Super complicated, inefficient branching and no good reason, not even then:

```
pub const fn escape(value: u8) -> u8 {
match value {
0xff => 0x6d,
0x7f => 0x6c,
0x10 | 0x90 | 0x11 | 0x91 | 0x13 | 0x93 | ZDLE => value ^ 0x40,
// Telenet command escaping, which actually necessary only when preceded
// by 0x40 or 0xc0, meaning that this could be optimized a bit with the
// help of previous byte.
0x0d | 0x8d => value ^ 0x40,
_ => value,
}
}
```

Overwriting data is the new "goto" sort of looking at more modern languages than C :-)

This is my ultimate goal with #tior: a multiplexed #TTY and file transfer through serial. It won't happen fast I'm barely gettting #ZMODEM together and need to finish the UI for my client after that.

Here I've carved up high-level goal what I have in mind: https://github.com/de-vri-es/serial2-rs/issues/6#issuecomment-1837024452

IMHO this is totally modern world thing. When I was still working e.g. with Nokia Linux phones security, earlier that on #Symbian, serial port was used a lot. Then it sort of phased out for a number of years in my work life but today with the maker community, all sorts of #SBC's and affordable #FPGA's the stocks are rising again. Of course if you've been all the time working on embedded, serial has been around for all this time. It is pretty weird that minicom and lrzsz still rock the world in that arena.

When you work on a a power on for a new hardware device, it is the first channel that gets usually enabled. And e.g. like any FPGA design or something you put to let's say to Arduino it is simplest route to get some communications on going.

So yeah, I really think this sort of endeavour does matter.

@epicEaston197 All cool. I don't mind. It was a bit culminated statement I made :-) I don't mind growing outreach but being like it is not bad either.

@epicEaston197 Fun fact: Google+ was hugely popular among kernel developers. That does not give a good prognosis for the success of Mastodon :-) For me it is not an issue tho because I care more about this as a productivity application than social media :-) Only social media account I've ever had has been Facebook (OK, couple of short spin offs to Twitter too) and that for the most part staying in contact with relatives.

I care exactly about Mastodon because social media madness, influencers etc. do not exist here. I'm happy that Mastodon is unpopular sort of TBH

So I did a pull request to the hex crate:

https://github.com/KokaKiwi/rust-hex/pull/83

This sort of scenario is not too uncommon in data flows especially when you want to squeeze everything bit out. E.g. for something like #ZMODEM you’d prefer the implementation to scale from microcontroller to Intel Xeon.

Usage example in my crate:

        if encoding == Encoding::ZHEX {
            hex::decode_in_slice(&mut out).or::<io::Error>(Err(ErrorKind::InvalidData.into()))?;
            out.truncate(out.len() / 2);
        }

#rustlang

E.g. if you have a hex string in stack, you could convert it to a binary to either head or tail of that memory block by overwriting it as you iterate. Further any data conversion where you convert any data from more sparse format to more compressed format.

E.g. I could have saved 7 bytes of stack by doing that sort of thing but do not want to do the effort because of inconvenience :-) Making world bloat again

One thing where #Rust is pretty inconvenient to use IMHO, when you want to optimize your code in a way that you want to modify data "within its space". I.e. purposely overwrite the same data chunk that you are iterating. Totally doable but not something you tend to do. Sort of thing that is against its design principles. #rustlang

trivial to get my #zmodem crate heapless with the `tinyvec` crate: https://github.com/jarkkojs/zmodem2/commit/6494028c3970c2c9f769643dac0e33c31a103fff

Still a few locations using heap but they are not more complicated than these sites :-)

As for "async" goes I think I just have a function to run one cycle of the state machine as it is fairly clean that way. It should be pretty easy then to wrap whatever framework is used around.

I.e. have start_send and next_send instead of just send and such and so forth. The way zmodem splits stuff that ZDATA subpackets fits well to this scheme. I should probably learn the language level async scheme but I see no point complicating this. I'm sure a framework using that can easily wrap a state machine.

#rustlang

One thing that I had to dig up from previous #Enarx work was core::marker::PhantomData. It is not so well known but pretty important concept inside Rust.

PhantomData is a zero-sized struct that merely acts as a life-time indicator for the other parameter, which are usually pointers on where it is applied. It is used to implement many of the core structs such as Rc to name one instance.

It is pretty good lesson on how lifetime parameters interact with the Rust compiler.

I’d even say that if you understand PhantomData, then you have the basic understanding of Rust, and if not, you still have to learn a bit. It is the block that the whole core library is based on after all.

#rustlang

I’d actually recommend to do few exercises with just from_raw_parts and lifetime parameters before using zerocopy because that sort of gives you full tutorial on what that particular crate does internally :-) I sort of enjoy thinking of those scenarios so it also takes some fun away :-)

Also this the Rust tutorial I wish I had found when I first started working on with Rust: https://google.github.io/comprehensive-rust/.

It is far better than any e.g. book I've ever seen on the topic of Rust.

At least this is by fart the best tutorial if you already have strong C/C++ and perhaps assembly background. Any books that I've ever read on Rust, well I've stop reading latest on chapter 2 because they have incredibly boring. This is really good hands on engineering material if you already sort of have understanding already topics such as memory management etc.

All the crates that #Google has done for #Rust seem to be like stuff I’ve been looking for to get better control of the memory.

Especially zerocopy is a time saver as it has all the thinkable stuff that I have used previously core::slice::from_raw_parts and spent a lot of time thinking of all the possible safety scenarios, such as this recent one:

impl<'a> From<&'a Header> for &'a [u8] {
    fn from(value: &Header) -> Self {
        // SAFETY: out-of-boundary is not possible, given that the size constraint
        // exists in the struct definition. The lifetime parameter links the lifetime
        // of the header reference to the slice.
        unsafe { from_raw_parts((value as *const Header) as *const u8, size_of::<Header>()) }
    }
}

Previously I’ve had to do similar consideration in the #Enarx project. You can do these by hand but it is nice to have a common crate, which is tested by many for these risky scenarios.

Other mentionable crate from Google is tinyvec, which I’m going to use in zmodem2 to remove internal heap usage.

@regehr Other than that I appreciate a lot a book called Embedded Linux Primer. It has all the essential pieces of Linux kernel binary de-construction and power on (initcalls, kernel command-line etc.).

@regehr Linkers and loader is by far the best and it has depth, and is just a great piece of text. And still fully valid. The more recent books are pretty dull compared to it, and more like tool guides than great prose.

I also have my own minicom etc. alike tool in early phases but do not yet want to advertise it too much:

https://github.com/jarkkojs/tior

The whole point starting to do this was to have a serial tool with great zmodem feature. Since a suitable crate did not exist I had to halt that and start working on this one.

Of course also for this pull requests are welcome. And I can promise to commit maintaining for some time as these are tools that I've needed in my work pretty much my whole career.

For text editors and email there's been always a great tool existing like vim and mutt for instance. For serial port, it was a downgrade to move on from MS-DOS to Linux, as minicom does not really compare to telex or telemate. And to this day there's nothing nearly as good as there were even for good old DOS.

So if you are interested of actually enabling ZMODEM for the Rust ecosystem and various applications dealing with ttyS, I'm open for contributions :-) I could see e.g. https://github.com/hacknus/serial-monitor-rust adapting this eventually. #rustlang

I made bunch of high-level issues to draw an idea what are my sort of immediate goals with this work so that it is easier to contribute given some (not too restrictive) goals have been set: https://github.com/jarkkojs/zmodem2/issues

first issue created for `zmodem2`: https://github.com/jarkkojs/zmodem2/issues/1

#zmodem #rustlang #tty