Skip to content

Commit

Permalink
Merge pull request #975 from mrcook/tzx
Browse files Browse the repository at this point in the history
tzx: Add suport for ZX Spectrum TZX and TAP files
  • Loading branch information
wader authored Aug 8, 2024
2 parents 4a74fbb + c93dc8b commit 7fca197
Show file tree
Hide file tree
Showing 15 changed files with 1,051 additions and 1 deletion.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,12 +141,14 @@ pssh_playready,
[rtmp](doc/formats.md#rtmp),
sll2_packet,
sll_packet,
[tap](doc/formats.md#tap),
tar,
tcp_segment,
tiff,
[tls](doc/formats.md#tls),
toml,
[tzif](doc/formats.md#tzif),
[tzx](doc/formats.md#tzx),
udp_datagram,
vorbis_comment,
vorbis_packet,
Expand Down
45 changes: 44 additions & 1 deletion doc/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,12 +113,14 @@
|[`rtmp`](#rtmp) |Real-Time&nbsp;Messaging&nbsp;Protocol |<sub>`amf0` `mpeg_asc`</sub>|
|`sll2_packet` |Linux&nbsp;cooked&nbsp;capture&nbsp;encapsulation&nbsp;v2 |<sub>`inet_packet`</sub>|
|`sll_packet` |Linux&nbsp;cooked&nbsp;capture&nbsp;encapsulation |<sub>`inet_packet`</sub>|
|[`tap`](#tap) |TAP&nbsp;tape&nbsp;format&nbsp;for&nbsp;ZX&nbsp;Spectrum&nbsp;computers |<sub></sub>|
|`tar` |Tar&nbsp;archive |<sub>`probe`</sub>|
|`tcp_segment` |Transmission&nbsp;control&nbsp;protocol&nbsp;segment |<sub></sub>|
|`tiff` |Tag&nbsp;Image&nbsp;File&nbsp;Format |<sub>`icc_profile`</sub>|
|[`tls`](#tls) |Transport&nbsp;layer&nbsp;security |<sub>`asn1_ber`</sub>|
|`toml` |Tom's&nbsp;Obvious,&nbsp;Minimal&nbsp;Language |<sub></sub>|
|[`tzif`](#tzif) |Time&nbsp;Zone&nbsp;Information&nbsp;Format |<sub></sub>|
|[`tzx`](#tzx) |TZX&nbsp;tape&nbsp;format&nbsp;for&nbsp;ZX&nbsp;Spectrum&nbsp;computers |<sub>`tap`</sub>|
|`udp_datagram` |User&nbsp;datagram&nbsp;protocol |<sub>`udp_payload`</sub>|
|`vorbis_comment` |Vorbis&nbsp;comment |<sub>`flac_picture`</sub>|
|`vorbis_packet` |Vorbis&nbsp;packet |<sub>`vorbis_comment`</sub>|
Expand All @@ -137,7 +139,7 @@
|`ip_packet` |Group |<sub>`icmp` `icmpv6` `tcp_segment` `udp_datagram`</sub>|
|`link_frame` |Group |<sub>`bsd_loopback_frame` `ether8023_frame` `ipv4_packet` `ipv6_packet` `sll2_packet` `sll_packet`</sub>|
|`mp3_frame_tags` |Group |<sub>`mp3_frame_vbri` `mp3_frame_xing`</sub>|
|`probe` |Group |<sub>`adts` `aiff` `apple_bookmark` `ar` `avi` `avro_ocf` `bitcoin_blkdat` `bplist` `bzip2` `caff` `elf` `fit` `flac` `gif` `gzip` `html` `jp2c` `jpeg` `json` `jsonl` `leveldb_table` `luajit` `macho` `macho_fat` `matroska` `moc3` `mp3` `mp4` `mpeg_ts` `nes` `ogg` `opentimestamps` `pcap` `pcapng` `png` `tar` `tiff` `toml` `tzif` `wasm` `wav` `webp` `xml` `yaml` `zip`</sub>|
|`probe` |Group |<sub>`adts` `aiff` `apple_bookmark` `ar` `avi` `avro_ocf` `bitcoin_blkdat` `bplist` `bzip2` `caff` `elf` `fit` `flac` `gif` `gzip` `html` `jp2c` `jpeg` `json` `jsonl` `leveldb_table` `luajit` `macho` `macho_fat` `matroska` `moc3` `mp3` `mp4` `mpeg_ts` `nes` `ogg` `opentimestamps` `pcap` `pcapng` `png` `tar` `tiff` `toml` `tzif` `tzx` `wasm` `wav` `webp` `xml` `yaml` `zip`</sub>|
|`tcp_stream` |Group |<sub>`dns_tcp` `rtmp` `tls`</sub>|
|`udp_payload` |Group |<sub>`dns`</sub>|

Expand Down Expand Up @@ -1209,6 +1211,26 @@ fq '.tcp_connections[] | select(.server.port=="rtmp") | d' file.cap
- https://rtmp.veriskope.com/docs/spec/
- https://rtmp.veriskope.com/pdf/video_file_format_spec_v10.pdf

## tap
TAP tape format for ZX Spectrum computers.

The TAP- (and BLK-) format is nearly a direct copy of the data that is stored
in real tapes, as it is written by the ROM save routine of the ZX-Spectrum.
A TAP file is simply one data block or a group of 2 or more data blocks, one
followed after the other. The TAP file may be empty.

You will often find this format embedded inside the TZX tape format.

The default file extension is `.tap`.

### Authors

- Michael R. Cook [email protected], original author

### References

- https://worldofspectrum.net/zx-modules/fileformats/tapformat.html

## tls
Transport layer security.

Expand Down Expand Up @@ -1378,6 +1400,27 @@ fq '.v2plusdatablock.leap_second_records | length' tziffile
### References
- https://datatracker.ietf.org/doc/html/rfc8536

## tzx
TZX tape format for ZX Spectrum computers.

`TZX` is a file format designed to preserve cassette tapes compatible with the
ZX Spectrum computers, although some specialized versions of the format have
been defined for other machines such as the Amstrad CPC and C64.

The format was originally created by Tomaz Kac, who was maintainer until
`revision 1.13`, before passing it to Martijn v.d. Heide. For a brief period
the company Ramsoft became the maintainers, and created revision `v1.20`.

The default file extension is `.tzx`.

### Authors

- Michael R. Cook [email protected], original author

### References

- https://worldofspectrum.net/TZXformat.html

## wasm
WebAssembly Binary Format.

Expand Down
3 changes: 3 additions & 0 deletions format/all/all.fqtest
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ $ fq -n _registry.groups.probe
"tar",
"tiff",
"tzif",
"tzx",
"wasm",
"webp",
"zip",
Expand Down Expand Up @@ -156,12 +157,14 @@ pssh_playready PlayReady PSSH
rtmp Real-Time Messaging Protocol
sll2_packet Linux cooked capture encapsulation v2
sll_packet Linux cooked capture encapsulation
tap TAP tape format for ZX Spectrum computers
tar Tar archive
tcp_segment Transmission control protocol segment
tiff Tag Image File Format
tls Transport layer security
toml Tom's Obvious, Minimal Language
tzif Time Zone Information Format
tzx TZX tape format for ZX Spectrum computers
udp_datagram User datagram protocol
vorbis_comment Vorbis comment
vorbis_packet Vorbis packet
Expand Down
2 changes: 2 additions & 0 deletions format/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,14 @@ import (
_ "github.com/wader/fq/format/protobuf"
_ "github.com/wader/fq/format/riff"
_ "github.com/wader/fq/format/rtmp"
_ "github.com/wader/fq/format/tap"
_ "github.com/wader/fq/format/tar"
_ "github.com/wader/fq/format/text"
_ "github.com/wader/fq/format/tiff"
_ "github.com/wader/fq/format/tls"
_ "github.com/wader/fq/format/toml"
_ "github.com/wader/fq/format/tzif"
_ "github.com/wader/fq/format/tzx"
_ "github.com/wader/fq/format/vorbis"
_ "github.com/wader/fq/format/vpx"
_ "github.com/wader/fq/format/wasm"
Expand Down
2 changes: 2 additions & 0 deletions format/format.go
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,14 @@ var (
RTMP = &decode.Group{Name: "rtmp"}
SLL_Packet = &decode.Group{Name: "sll_packet"}
SLL2_Packet = &decode.Group{Name: "sll2_packet"}
TAP = &decode.Group{Name: "tap"}
TAR = &decode.Group{Name: "tar"}
TCP_Segment = &decode.Group{Name: "tcp_segment"}
TIFF = &decode.Group{Name: "tiff"}
TLS = &decode.Group{Name: "tls"}
TOML = &decode.Group{Name: "toml"}
Tzif = &decode.Group{Name: "tzif"}
TZX = &decode.Group{Name: "tzx"}
UDP_Datagram = &decode.Group{Name: "udp_datagram"}
Vorbis_Comment = &decode.Group{Name: "vorbis_comment"}
Vorbis_Packet = &decode.Group{Name: "vorbis_packet"}
Expand Down
158 changes: 158 additions & 0 deletions format/tap/tap.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package tzx

// https://worldofspectrum.net/zx-modules/fileformats/tapformat.html

import (
"bufio"
"bytes"
"embed"

"golang.org/x/text/encoding/charmap"

"github.com/wader/fq/format"
"github.com/wader/fq/pkg/bitio"
"github.com/wader/fq/pkg/decode"
"github.com/wader/fq/pkg/interp"
"github.com/wader/fq/pkg/scalar"
)

//go:embed tap.md
var tapFS embed.FS

func init() {
interp.RegisterFormat(
format.TAP,
&decode.Format{
Description: "TAP tape format for ZX Spectrum computers",
DecodeFn: tapDecoder,
})

interp.RegisterFS(tapFS)
}

// The TAP- (and BLK-) format is nearly a direct copy of the data that is stored
// in real tapes, as it is written by the ROM save routine of the ZX-Spectrum.
// A TAP file is simply one data block or a group of 2 or more data blocks, one
// followed after the other. The TAP file may be empty.
func tapDecoder(d *decode.D) any {
d.Endian = decode.LittleEndian

d.FieldArray("blocks", func(d *decode.D) {
for !d.End() {
d.FieldStruct("block", func(d *decode.D) {
decodeTapBlock(d)
})
}
})
return nil
}

func decodeTapBlock(d *decode.D) {
// Length of the following data.
length := d.FieldU16("length")

// read header, fragment, or data block
switch length {
case 0:
// fragment with no data
case 1:
d.FieldRawLen("data", 8)
case 19:
d.FieldStruct("header", func(d *decode.D) {
decodeHeader(d)
})
default:
d.FieldStruct("data", func(d *decode.D) {
decodeDataBlock(d, length)
})
}
}

// decodes the different types of 19-byte header blocks.
func decodeHeader(d *decode.D) {
blockStartPosition := d.Pos()

// Always 0: byte indicating a standard ROM loading header
d.FieldU8("flag", scalar.UintMapSymStr{0: "standard_speed_data"})
// Header type
dataType := d.FieldU8("data_type", scalar.UintMapSymStr{
0x00: "program",
0x01: "numeric",
0x02: "alphanumeric",
0x03: "data",
})
// Loading name of the program. Filled with spaces (0x20) to 10 characters.
d.FieldStr("program_name", 10, charmap.ISO8859_1)

switch dataType {
case 0:
// Length of data following the header = length of BASIC program + variables.
d.FieldU16("data_length")
// LINE parameter of SAVE command. Value 32768 means "no auto-loading".
// 0..9999 are valid line numbers.
d.FieldU16("auto_start_line")
// Length of BASIC program;
// remaining bytes ([data length] - [program length]) = offset of variables.
d.FieldU16("program_length")
case 1:
// Length of data following the header = length of number array * 5 + 3.
d.FieldU16("data_length")
// Unused byte.
d.FieldU8("unused0")
// (1..26 meaning A..Z) + 128.
d.FieldU8("variable_name", scalar.UintHex)
// UnusedWord: 32768.
d.FieldU16("unused1")
case 2:
// Length of data following the header = length of string array + 3.
d.FieldU16("data_length")
// Unused byte.
d.FieldU8("unused0")
// (1..26 meaning A$..Z$) + 192.
d.FieldU8("variable_name", scalar.UintHex)
// UnusedWord: 32768.
d.FieldU16("unused1")
case 3:
// Length of data following the header, in case of a SCREEN$ header = 6912.
d.FieldU16("data_length")
// In case of a SCREEN$ header = 16384.
d.FieldU16("start_address", scalar.UintHex)
// UnusedWord: 32768.
d.FieldU16("unused")
default:
d.Fatalf("invalid TAP header type, got: %d", dataType)
}

// Simply all bytes XORed (including flag byte).
d.FieldU8("checksum", d.UintValidate(calculateChecksum(d, blockStartPosition, d.Pos()-blockStartPosition)), scalar.UintHex)
}

func decodeDataBlock(d *decode.D, length uint64) {
blockStartPosition := d.Pos()

// flag indicating the type of data block, usually 255 (standard speed data)
d.FieldU8("flag", scalar.UintFn(func(s scalar.Uint) (scalar.Uint, error) {
if s.Actual == 0xFF {
s.Sym = "standard_speed_data"
} else {
s.Sym = "custom_data_block"
}
return s, nil
}))
// The essential data: length minus the flag/checksum bytes (may be empty)
d.FieldRawLen("data", int64(length-2)*8)
// Simply all bytes (including flag byte) XORed
d.FieldU8("checksum", d.UintValidate(calculateChecksum(d, blockStartPosition, d.Pos()-blockStartPosition)), scalar.UintHex)
}

func calculateChecksum(d *decode.D, blockStartPos, blockEndPos int64) uint64 {
var blockSlice bytes.Buffer
writer := bufio.NewWriter(&blockSlice)
d.Copy(writer, bitio.NewIOReader(d.BitBufRange(blockStartPos, blockEndPos)))

var checksum uint8
for _, v := range blockSlice.Bytes() {
checksum ^= v
}
return uint64(checksum)
}
16 changes: 16 additions & 0 deletions format/tap/tap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
The TAP- (and BLK-) format is nearly a direct copy of the data that is stored
in real tapes, as it is written by the ROM save routine of the ZX-Spectrum.
A TAP file is simply one data block or a group of 2 or more data blocks, one
followed after the other. The TAP file may be empty.

You will often find this format embedded inside the TZX tape format.

The default file extension is `.tap`.

### Authors

- Michael R. Cook [email protected], original author

### References

- https://worldofspectrum.net/zx-modules/fileformats/tapformat.html
22 changes: 22 additions & 0 deletions format/tap/testdata/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
### basic_prog1.tap

The `basic_prog1.tap` test file was created directory from the FUSE emulator.

Inside the emulated ZX Spectrum a BASIC program was created:

```
10 PRINT "fq is the best!"
20 GOTO 10
```

and saved to tape:

```
SAVE "fqTestProg", LINE 10
```

Then from FUSE select the menu item `Media > Tape > Save As..`.

Any BASIC, machine code, screen image, or other data, can be saved directly
using the `SAVE` command. Further instructions can be found here:
https://worldofspectrum.org/ZXBasicManual/zxmanchap20.html
21 changes: 21 additions & 0 deletions format/tap/testdata/basic_prog1.fqtest
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
$ fq -d tap dv basic_prog1.tap
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: basic_prog1.tap (tap) 0x0-0x3f (63)
| | | blocks[0:2]: 0x0-0x3f (63)
| | | [0]{}: block 0x0-0x15 (21)
0x00|13 00 |.. | length: 19 0x0-0x2 (2)
| | | header{}: 0x2-0x15 (19)
0x00| 00 | . | flag: "standard_speed_data" (0) 0x2-0x3 (1)
0x00| 00 | . | data_type: "program" (0) 0x3-0x4 (1)
0x00| 66 71 54 65 73 74 50 72 6f 67 | fqTestProg | program_name: "fqTestProg" 0x4-0xe (10)
0x00| 26 00| &.| data_length: 38 0xe-0x10 (2)
0x10|0a 00 |.. | auto_start_line: 10 0x10-0x12 (2)
0x10| 26 00 | &. | program_length: 38 0x12-0x14 (2)
0x10| 01 | . | checksum: 0x1 (valid) 0x14-0x15 (1)
| | | [1]{}: block 0x15-0x3f (42)
0x10| 28 00 | (. | length: 40 0x15-0x17 (2)
| | | data{}: 0x17-0x3f (40)
0x10| ff | . | flag: "standard_speed_data" (255) 0x17-0x18 (1)
0x10| 00 0a 14 00 20 f5 22 66| .... ."f| data: raw bits 0x18-0x3e (38)
0x20|71 20 69 73 20 74 68 65 20 62 65 73 74 21 22 0d|q is the best!".|
0x30|00 14 0a 00 ec 31 30 0e 00 00 0a 00 00 0d |.....10....... |
0x30| b6| | .|| checksum: 0xb6 (valid) 0x3e-0x3f (1)
Binary file added format/tap/testdata/basic_prog1.tap
Binary file not shown.
28 changes: 28 additions & 0 deletions format/tzx/testdata/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
### basic_prog1.tzx

The `basic_prog1.tzx` test file was created directory from the FUSE emulator.

Inside the emulated ZX Spectrum a BASIC program was created:

```
10 PRINT "fq is the best!"
20 GOTO 10
```

and saved to tape:

```
SAVE "fqTestProg", LINE 10
```

Then from FUSE select the menu item `Media > Tape > Save As..`.

Any BASIC, machine code, screen image, or other data, can be saved directly
using the `SAVE` command. Further instructions can be found here:
https://worldofspectrum.org/ZXBasicManual/zxmanchap20.html


#### Archive Info

The FUSE emulator is not able to add the tape metadata. As this tape block is
very simple, it was added manually using a Hex editor.
Loading

0 comments on commit 7fca197

Please sign in to comment.