Eyes are more sensible to light (gradient) changes, so with an HAM/YUV mode if the first "change command" is related to the Y channel, it'll make quite less artifacts.

May be a test program will prove it.

Woitjek and others: Lossless image compression can get you about 1:2 as compression rate, not more. There are simple hardware-suitable algorithms for that, for example JPEG-LS would do that. But in either case, despite being the JPEG guy here, I would *not* recommend doing that. It adds another level of complexity that is quite unnecessary, and the decompression would better be done in software if it is needed by the program, instead being done invisibly to the system by some hardware.

Greetings from Newark,

Thomas
 

So, I see no advantage on having the decompression made buy a second CPU: the main one can do the work as well.

Most of the time this unit will be unused.  While it is not being used for compression/decompression it can be used for any other thing imaginible.

Sure.

A hardcoded FPGA codec would be very kewl but it is just so unflexible.  We get stuck with just 1 algorithm.  When we discover a better algorithm in the future?  Oh well, we are stuck with an old hardcoded algorithm. :(

We don't know how the NOVA algorithm works, so you can't say that.

From Gunnar's words, it seems pretty easy and very cost effective for the FPGA.

ZIP's deflat algoritm will require a lot more resources, since it involves both LZ77 + Huffman, and proper buffers to be used.

So I vote for using a 2nd 68070 to do the job.  Like Wojtek said, we can leave out some lame instructions such as BCD, if that helps.

I don't know how much space is required for the BCDs instructions, but if they are cheap, there's no need to change the ISA for the second 68K processor.

Anyway, if the ISA have to simplified, we can remove:
- BCDs;
- CHK/CHK2/CAS/CAS2;
- MOVEP;
- double indirect address modes.

And I am sure that in the first Natami we would probably have to greatly reduce its cache size.  But a few years down the road when better FPGA's are available then we can increase the cache fpr faster performance and 100% compatibility.

Yup.

This boils down to the same argument as polling IO or DMA.

If you have a seperate unit (either a DMA FPGA engine or a second CPU) doing the loading of the disk-data and the decompression for you then you can load a next level in parallel while playing the game.

The same is true with AMIGA OS and the Workbench.
Your main CPU (remember AMIGA OS only supports 1 CPU!) is free to run AMIGA OS - while the acceleration Unit (again it does not matter whether this is a CPU or special Unit) will do the copy work.

This means the "IO-acceleration" will at the end of the day off-load the core CPU.
This is very much the same situation that SCSI versus ATA was in the old days.

But of course you can.
You know XPK, don't you?
Its just the same concept.
 
If you don't know XPK think of Powerpacker and PPlib.

Greetings (now from Boston),

Thomas

So the CPU will have to wait in any case, and the advantage is too little.

The same is true with AMIGA OS and the Workbench.   Your main CPU (remember AMIGA OS only supports 1 CPU!) is free to run AMIGA OS - while the acceleration Unit (again it does not matter whether this is a CPU or special Unit) will do the copy work.     This means the "IO-acceleration" will at the end of the day off-load the core CPU.   This is very much the same situation that SCSI versus ATA was in the old days.

That's an interesting scenery, but you have the CPU to do something else while the system is decompressing datas.

Usually, you know, a human being works with a single (main) task at the time.

When I launch a new application, I wait it to be loaded and executed, then I'm interested on working with it.

So, I must wait anyway. That's the primary point for a user.

The module will just use the decompression hardware when needed, instead of the CPU, to accelerate the operation.

I just don't see why a highly valuable resource like a real CPU should do this very low-level stuff, which can be integrated seamlessly into the DMA (switchable). Doing a MP3 decoder, or game AI etc with a subcore CPU, makes sense, IMO. But not this type of work.

Think about it - this tiny, realtime hardware decompressor accelerates the system in general. It is not about saving space, obviously. Who thinks that saving storage space is the motivation, has misunderstood the whole discussion. Data is best, when you don't have to transmit it. I think we all agree on this,especially on out not so super fast memory interfaces.

A fully-fledged CPU is a incredibly scarce and expensive resource in a system. If you use it all the time for grunt work, you are blocking this valuable resource, which could otherwise do complex, high-level work which makes no sense to directly implement a special solution for it in hardware. Using a CPU for grunt work lowers overall system speed, no matter whether it's the main CPU or a helper CPU.

In Natami, we must find ALL possible ways to accelerate the overall system speed, at the lowest possible price in logic resources. Adding an experimental tiny decompressor, doesn't hurt. Even if someone finds it useless, then this is one opinion of many many others. If the majority finds it actually not worthwhile, then we can remove this experimental feature again. As long as it is marked as experimental and people know it could possibly be removed again in the future, they know what they are dealing with. Natami is one great experimenting platform, for a major part. If we don't do experimental things here and there, we can stop right now and buy some Intel chips.

OK Wojtek, I understand. You will not use a decompression hardware.
That is fair and OK for me.
But then all further comments on this topic from your side are obsolete as HW decompression no longer affects you. I don't need to read them. Others don't need to read them. So you can save your time not writing them.

PS: Sorry I could not resist feeding him. I am no good in acting as role model. :)

It's better to have 3-plane 8-bit YUV with U,V planes resolution halved.

already proved.

Actually there is no need for extra CPU to be 68k.

+1
I realy like the idea. New applications developed for the Natami will most likly grow in size(/me throwing oil on the fire). for several reasons. we have more mem and cpu. standards like xml/html are well spread. using more script languages like python etc. these "standards" use more space due to the text vs binary. and larger files take more time to load. Maybe the speed increase for games will not be that much(when using already compressed data standards) but applications can certainly benefit a lot.

I don't really see that this is much of a speed problem that requires hardware support, or should require hardware support. For example, if a PP compatible packer is implemented in hardware, would I use it? Probably not, there are better algorithms than PP, and zip would be more versatile. Yet, zip is not suitable for images, and bz2 is a better universal algorithm. For audio, I would neither use bz2 nor zip, but some lossy code...

Instead of implementing a specific hardware algorithm, I would rather look at which CPU instructions would be beneficial for compression. This is mostly bitjuggling, searching and table-lookup type of instructions, low-level entropy coding stuff (huffman, arithmetic coding) and then leave it to software to use this type of hardware support to implement specific codecs.

Greetings from Boston,

Thomas

quick test - 2GB files compressed to 700MB with gzip -9
decompressing it from /tmp (ramdisk) under unix takes 13 seconds, so 140MB/s

  CPU: Pentium(R) Dual-Core CPU      T4500  @ 2.30GHz (2294.53-MHz K8-class CPU)

gzip is C code, no asssembly. how much slower would be n68k with code that do lots of random L1 cache accesses and non-predictable branches - huffman decode+copy blocks.

i bet 10MB/s by just compiling gzip with gcc, 20MB/s with assembly.

As for bzip2 it's BAD unless you have megabytes of cache.
grzip is too but much less bad and compresses actully far better than bzip2. This is the packer i use under unix when i really care of size not time - for eg before transmitting file over slow connection.

EXTERNAL LINK 
Porting is trivial. There is one .c file - compressor library and other handling command line and file i/o

i REALLY recommend it.
It packs always faster than bzip2, unpack with similar speed, and always packs better.

there are options to select how it works, in my experiments -b8m -m3 gives best results.