One thing where #Rust is pretty inconvenient to use IMHO, when you want to optimize your code in a way that you want to modify data "within its space". I.e. purposely overwrite the same data chunk that you are iterating. Totally doable but not something you tend to do. Sort of thing that is against its design principles. #rustlang

trivial to get my #zmodem crate heapless with the `tinyvec` crate: https://github.com/jarkkojs/zmodem2/commit/6494028c3970c2c9f769643dac0e33c31a103fff

Still a few locations using heap but they are not more complicated than these sites :-)

As for "async" goes I think I just have a function to run one cycle of the state machine as it is fairly clean that way. It should be pretty easy then to wrap whatever framework is used around.

I.e. have start_send and next_send instead of just send and such and so forth. The way zmodem splits stuff that ZDATA subpackets fits well to this scheme. I should probably learn the language level async scheme but I see no point complicating this. I'm sure a framework using that can easily wrap a state machine.

#rustlang

One thing that I had to dig up from previous #Enarx work was core::marker::PhantomData. It is not so well known but pretty important concept inside Rust.

PhantomData is a zero-sized struct that merely acts as a life-time indicator for the other parameter, which are usually pointers on where it is applied. It is used to implement many of the core structs such as Rc to name one instance.

It is pretty good lesson on how lifetime parameters interact with the Rust compiler.

I’d even say that if you understand PhantomData, then you have the basic understanding of Rust, and if not, you still have to learn a bit. It is the block that the whole core library is based on after all.

#rustlang

All the crates that #Google has done for #Rust seem to be like stuff I’ve been looking for to get better control of the memory.

Especially zerocopy is a time saver as it has all the thinkable stuff that I have used previously core::slice::from_raw_parts and spent a lot of time thinking of all the possible safety scenarios, such as this recent one:

impl<'a> From<&'a Header> for &'a [u8] {
    fn from(value: &Header) -> Self {
        // SAFETY: out-of-boundary is not possible, given that the size constraint
        // exists in the struct definition. The lifetime parameter links the lifetime
        // of the header reference to the slice.
        unsafe { from_raw_parts((value as *const Header) as *const u8, size_of::<Header>()) }
    }
}

Previously I’ve had to do similar consideration in the #Enarx project. You can do these by hand but it is nice to have a common crate, which is tested by many for these risky scenarios.

Other mentionable crate from Google is tinyvec, which I’m going to use in zmodem2 to remove internal heap usage.

first issue created for `zmodem2`: https://github.com/jarkkojs/zmodem2/issues/1

#zmodem #rustlang #tty

#zmodem2 is a nice history lesson to develop:

    style: cleanup and fix cosmetic stuff

    1. This inherits from original `zmodem` crate: "ZLDE" is in-fact ZDLE,
       an acronym of "ZMODEM Data Link Escape" character.
    2. Fine-tune use statements.

    Link: https://wiki.synchro.net/ref:zmodem
    Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@iki.fi>

That link in the commit message is a great source of information on #zmodem.

#rustlang

next step in my zmodem2 crate is to make send and receive asynchronous so that progress can be tracked. then i’m ready to integrate this to my other small project, tior. #zmodem #rustlang

converted legacy hard coded test cases for frame to rstest in the #zmodem 2 crate:

#[cfg(test)]
mod tests {
    use crate::frame::*;

    #[rstest::rstest]
    #[case(Encoding::ZBIN, Type::ZRQINIT, &[ZPAD, ZLDE, Encoding::ZBIN as u8, 0, 0, 0, 0, 0, 0, 0])]
    #[case(Encoding::ZBIN32, Type::ZRQINIT, &[ZPAD, ZLDE, Encoding::ZBIN32 as u8, 0, 0, 0, 0, 0, 29, 247, 34, 198])]
    fn test_header(
        #[case] encoding: Encoding,
        #[case] frame_type: Type,
        #[case] expected: &[u8]
    ) {
        let header = Header::new(encoding, frame_type);

        let mut packet = vec![];
        new_frame(&header, &mut packet);

        assert_eq!(packet, expected);
    }
    #[rstest::rstest]
    #[case(Encoding::ZBIN, Type::ZRQINIT, &[1, 1, 1, 1], &[ZPAD, ZLDE, Encoding::ZBIN as u8, 0, 1, 1, 1, 1, 98, 148])]
    #[case(Encoding::ZHEX, Type::ZRQINIT, &[1, 1, 1, 1], &[ZPAD, ZPAD, ZLDE, Encoding::ZHEX as u8, b'0', b'0', b'0', b'1', b'0', b'1', b'0', b'1', b'0', b'1', 54, 50, 57, 52, b'\r', b'\n', XON])]
    fn test_header_with_flags(
        #[case] encoding: Encoding,
        #[case] frame_type: Type,
        #[case] flags: &[u8; 4],
        #[case] expected: &[u8]
    ) {
        let header = Header::new(encoding, frame_type).flags(flags);

        let mut packet = vec![];
        new_frame(&header, &mut packet);

        assert_eq!(packet, expected);
    }
}

Should be easier to refactor the legacy code now as there is less raw code that might be affected in tests.

#rustlang #rstest

I hope I got this right (safety-proprty), i.e. so that references are enforced to have equal life-time:

impl<'a> From<&'a Header> for &'a [u8] {
    fn from(value: &Header) -> Self {
        // SAFETY: out-of-boundary is not possible, given that the size constraint
        // exists in the struct definition. The lifetime parameter links the lifetime
        // of the header reference to the slice.
        unsafe { from_raw_parts((value as *const Header) as *const u8, size_of::<Header>()) }
    }
}

#rustlang

great implemented transmutable (with u8) enum for the frame type of the #ZMODEM transfer protocol header: https://github.com/jarkkojs/zmodem2/commit/c76316c2ecae097be03506e8ce19d61287e6468a

I can use this as a reference model for refactoring rest of the code base.

Next, I’ll replace encoding: u8 with a similar enum Encoding .

#rustlang

Known good crates for no_std Rust:

1. spinning (used e.g. in #Enarx)
2. heapless

Will be updated over time.

#note #rustlang

After trial and error, i.e. brute force going through crc crate algorithms, I can say that CRC32_32_ISO_HDLC is the correct variant for ZMODEM 32-bit transfers, i.e. cipher can be acquired by:

const CRC32: Crc<u32> = Crc::<u32>::new(&CRC_32_ISO_HDLC);

It is better to declare like this so that it gets compiled into .rodata and not initialized at run-time.

I really like this heapless. It sort of helps to implement my strategy for developing Rust programs:

1.Maximize no_std surface. 2 Minimize heap allocations.

It is easier to see then the hot zones where the program actually dynamically grows for a good reason, similarly as with unsafe blocks it is easy to the red zones for memory errors. This helps a lot with availability and protection against denial-of-service (DoS) attacks.

So to summarize I don’t split Rust program in my mind just to “unsafe” and “safe” but instead I split it “unsafe”, “static” and “dynamic”, or along the lines.

really like this fsmetry crate (also no_std):

fsmentry::dsl! {
    #[derive(Debug)]
    pub Mode {
        WaitingInput -> WaitingCommand -> WaitingInput;
        WaitingCommand -> SendingFile -> WaitingInput;
        WaitingCommand -> ReceivingFile -> WaitingInput;
        WaitingCommand -> Exit;
    }
}

I use it manage life-cycle in my small serial port tool tior. I also have some #zmodem code together but it is apparently much bigger leap to implement the #cli interface than it is to implement the protocol. I had to take some time to refactor existing code (e.g. to put FSM in place) and now I’m doing file path auto-completing interface for sending and receiving files with zmodem.

For the text input I’m going to use inquire.

I guess the definition of feature complete for 0.1 version is fully working zmodem transfers and known bugs have been fixed. Right now there is a single known bug: https://github.com/jarkkojs/tior/issues/1.

#tior #fsmentry #inquire #rustlang

in zmodem protocol there's lot of places where you could "low-barrier" modernize it and i think i'm going after that. like even though there is binary transfer the receiver responds always with hex string for ack's. so that doubles amount of bandwidth for no good.

I've started to use #himalaya as my email client. I switched from mutt to aerc year ago but it has not worked for my intuition but this actually feels a fresh take on #email.

What makes it a fresh take is in my opinion how it reduces the amount of context switching when you have email sort of integrated to the shell. In mutt you can do a lot without leaving the client but this sort of takes away the whole issue and gives full shell access.

software products of finnish software company arisoft. it is fun observation that the whole serial link thing that they had on-going is generation over what people use today when they connect TTL-USB-cable. Arisoft had a MS-DOS software in the early 90s that made possible to "smurf" your way in through the serial and have simultaneous file transfer at the same time. multitasking is imho the future.

so when i look at this web page i'm literally not looking into the past but to one generation ahead in the hopefully not so distant future.

#zmodem #smodem #bbs #serial #tty

it is 2023 and i'm implementing #zmodem just a sub-portion of it: 32-bit binary transfers. So ignoring 16-bit and alternative hex protocol. There's some non lively looking zmodem crate and of course lrzsz but neither of them feel too inspiring to contribute or build on top off.

all sort of glitch with #rustlang community comes to personally in the end how you look at a programming language: a language or purely just an opcode generator. one example of such glitch is IMHO unrobust handling of OOM conditions. So you end up getting safe but and super fat and resource wasteful user space programs. But still memory-safe in the language level, which does not provide safety for e.g. any possible DoS attack.

tonights jam, something in progress #bitwig