All the crates that #Google has done for #Rust seem to be like stuff I’ve been looking for to get better control of the memory.

Especially zerocopy is a time saver as it has all the thinkable stuff that I have used previously core::slice::from_raw_parts and spent a lot of time thinking of all the possible safety scenarios, such as this recent one:

impl<'a> From<&'a Header> for &'a [u8] {
    fn from(value: &Header) -> Self {
        // SAFETY: out-of-boundary is not possible, given that the size constraint
        // exists in the struct definition. The lifetime parameter links the lifetime
        // of the header reference to the slice.
        unsafe { from_raw_parts((value as *const Header) as *const u8, size_of::<Header>()) }
    }
}

Previously I’ve had to do similar consideration in the #Enarx project. You can do these by hand but it is nice to have a common crate, which is tested by many for these risky scenarios.

Other mentionable crate from Google is tinyvec, which I’m going to use in zmodem2 to remove internal heap usage.

@regehr Other than that I appreciate a lot a book called Embedded Linux Primer. It has all the essential pieces of Linux kernel binary de-construction and power on (initcalls, kernel command-line etc.).

@regehr Linkers and loader is by far the best and it has depth, and is just a great piece of text. And still fully valid. The more recent books are pretty dull compared to it, and more like tool guides than great prose.

I also have my own minicom etc. alike tool in early phases but do not yet want to advertise it too much:

https://github.com/jarkkojs/tior

The whole point starting to do this was to have a serial tool with great zmodem feature. Since a suitable crate did not exist I had to halt that and start working on this one.

Of course also for this pull requests are welcome. And I can promise to commit maintaining for some time as these are tools that I've needed in my work pretty much my whole career.

For text editors and email there's been always a great tool existing like vim and mutt for instance. For serial port, it was a downgrade to move on from MS-DOS to Linux, as minicom does not really compare to telex or telemate. And to this day there's nothing nearly as good as there were even for good old DOS.

So if you are interested of actually enabling ZMODEM for the Rust ecosystem and various applications dealing with ttyS, I'm open for contributions :-) I could see e.g. https://github.com/hacknus/serial-monitor-rust adapting this eventually. #rustlang

I made bunch of high-level issues to draw an idea what are my sort of immediate goals with this work so that it is easier to contribute given some (not too restrictive) goals have been set: https://github.com/jarkkojs/zmodem2/issues

first issue created for `zmodem2`: https://github.com/jarkkojs/zmodem2/issues/1

#zmodem #rustlang #tty

My crate has the implementation now in 933 lines and not a lot of dependencies. It starts to be in a shape that is not a huge stretch to make it fully no_std even. Not the first priority tho but entirely possible.

After I have a similar state function for the receiver I’ll look at futures crate, which claims to be no_std compatible. I.e. idea would be to get yield once per iteration.

I’m not interested in async stuff for multi-threading. I’m interested single-threaded sequenced type of stuff, co-operative multitasking…

This should clarify how it is organized:

/// Map the previous frame type of the sender and incoming frame type of the
/// receiver to the next packet to be sent.
///
/// NOTE: ZRINIT is used here as a wait state, as the sender does not use it for
/// other purposes. Other than tat the states map to the packets that the sender
/// sends next.
const fn next_state(sender: Option<Type>, receiver: Type) -> Option<Type> {
    match (sender, receiver) {
        (None, Type::ZRINIT) => Some(Type::ZFILE),
        (None, _) => Some(Type::ZRQINIT),
        (Some(Type::ZRQINIT), Type::ZRINIT) => Some(Type::ZFILE),
        (Some(Type::ZFILE), Type::ZRPOS) => Some(Type::ZDATA),
        (Some(Type::ZFILE), Type::ZRINIT) => Some(Type::ZRINIT),
        (Some(Type::ZRINIT), Type::ZRPOS) => Some(Type::ZDATA),
        (Some(Type::ZDATA), Type::ZACK) => Some(Type::ZDATA),
        (Some(Type::ZDATA), Type::ZRPOS) => Some(Type::ZDATA),
        (Some(Type::ZDATA), Type::ZRINIT) => Some(Type::ZFIN),
        (Some(Type::ZFIN), Type::ZFIN) => None,
        (_, _) => None,
    }
}

No reason to add complexity with fsmentry. This is a nice clean const function.

#zmodem2 is a nice history lesson to develop:

    style: cleanup and fix cosmetic stuff

    1. This inherits from original `zmodem` crate: "ZLDE" is in-fact ZDLE,
       an acronym of "ZMODEM Data Link Escape" character.
    2. Fine-tune use statements.

    Link: https://wiki.synchro.net/ref:zmodem
    Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@iki.fi>

That link in the commit message is a great source of information on #zmodem.

#rustlang

After looking that for some time my state diagram for “sz” is simply this:

fsmentry::dsl! {
    #[derive(Debug)]
    pub Mode {
        WaitingForReceiver -> Starting;
        Starting -> Sending;
        Starting -> WaitingForPosition;
        Sending -> WaitingForPosition;
        WaitingForPosition -> Sending;
        Sending -> Ending;
    }
}

This is scales exactly of the normal serial transfer. ZCHALLENGE/ZCOMPL, “corporate zmodem” (yes it was a real concept back in the say) and running commands by the request of the sender are definitely out of scope. So pretty clean and understandable in the end when you carve deep enough…

Moving file through the serial post is tricky, not because of protocols but because mostly it is based on launching rz and sz. This indirection makes impossible to implement an user experience that people used to transferring file such as showing progress, canceling the transfer and such and so forth. I'm fed up with this difficulty so that's where the motivation comes from.

I was going to use `fsmentry` crate for the state machine but then i ended up to something way more simpler: https://github.com/jarkkojs/zmodem2/commit/86a4af491448cf30a718a43dd1e5998b5b1863c6

next step in my zmodem2 crate is to make send and receive asynchronous so that progress can be tracked. then i’m ready to integrate this to my other small project, tior. #zmodem #rustlang

@Dr_Emann Yeah, I was not doing that because I can but because I had to :-) Good lesson anyway with the type system, learned a lot even though the snippet is only few lines.

@Dr_Emann This was a good tip. I want to refactor this as a zero-copy thing overally but doing these primitives takes time because you really have to think through, I was going to add cast from bytes next. Saves time!

@Dr_Emann thanks i took the advice and migrated! https://github.com/jarkkojs/zmodem2/commit/b374f24d9666ed7379b74047a556afc1a508cc55

converted legacy hard coded test cases for frame to rstest in the #zmodem 2 crate:

#[cfg(test)]
mod tests {
    use crate::frame::*;

    #[rstest::rstest]
    #[case(Encoding::ZBIN, Type::ZRQINIT, &[ZPAD, ZLDE, Encoding::ZBIN as u8, 0, 0, 0, 0, 0, 0, 0])]
    #[case(Encoding::ZBIN32, Type::ZRQINIT, &[ZPAD, ZLDE, Encoding::ZBIN32 as u8, 0, 0, 0, 0, 0, 29, 247, 34, 198])]
    fn test_header(
        #[case] encoding: Encoding,
        #[case] frame_type: Type,
        #[case] expected: &[u8]
    ) {
        let header = Header::new(encoding, frame_type);

        let mut packet = vec![];
        new_frame(&header, &mut packet);

        assert_eq!(packet, expected);
    }
    #[rstest::rstest]
    #[case(Encoding::ZBIN, Type::ZRQINIT, &[1, 1, 1, 1], &[ZPAD, ZLDE, Encoding::ZBIN as u8, 0, 1, 1, 1, 1, 98, 148])]
    #[case(Encoding::ZHEX, Type::ZRQINIT, &[1, 1, 1, 1], &[ZPAD, ZPAD, ZLDE, Encoding::ZHEX as u8, b'0', b'0', b'0', b'1', b'0', b'1', b'0', b'1', b'0', b'1', 54, 50, 57, 52, b'\r', b'\n', XON])]
    fn test_header_with_flags(
        #[case] encoding: Encoding,
        #[case] frame_type: Type,
        #[case] flags: &[u8; 4],
        #[case] expected: &[u8]
    ) {
        let header = Header::new(encoding, frame_type).flags(flags);

        let mut packet = vec![];
        new_frame(&header, &mut packet);

        assert_eq!(packet, expected);
    }
}

Should be easier to refactor the legacy code now as there is less raw code that might be affected in tests.

#rustlang #rstest

I hope I got this right (safety-proprty), i.e. so that references are enforced to have equal life-time:

impl<'a> From<&'a Header> for &'a [u8] {
    fn from(value: &Header) -> Self {
        // SAFETY: out-of-boundary is not possible, given that the size constraint
        // exists in the struct definition. The lifetime parameter links the lifetime
        // of the header reference to the slice.
        unsafe { from_raw_parts((value as *const Header) as *const u8, size_of::<Header>()) }
    }
}

#rustlang

I first used transmute but that causes extra struct copy.