| Stream Decompression | page 1 2 3 4 5 6 7 8
|
|---|
|
|---|
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
02 Feb 2011 08:38
| We could add some simple stream decompression to the NATAMI.
I think this could give a nice IO performance boost.
Normally programs on disk do compress fairly well.
If the stream decompression is on the fly, then it will not add any overhead.Example:
If we put a NOVA de-compressor into the FPGA then we could decompress about 200MB/sec on the fly.
Lets say you have a MemoryCard of 16GB capacity and 20MB/sec peak transfer speed. With streaming decompression the card would feel as about a 32GB card with 40 MB/sec transfer rate. Or a harddrive with 40 MB/sec transfer speed this could feel having 80MB/Sec transfer speed. Of course the mileage will vary. If you use lots of uncompressed applications (normal AMIGA apps, icons etc) then doubling the speed is realistic. Of course for already heavy compressed items like mp3 or movies the gain will be much less.
| |
Thierry Atheist
Canada
| |
Posts 1828
02 Feb 2011 08:56
| Sounds wicked awesome!!! As long as it's not wicked awful, as microsoft's "double space" or whatever it was.... Could it be made so as to pass on compressing and decompressing already compressed things such as dms, xpk, rar, lha, zip, mp3, MPEG, powerpacker, etc., as it would be useless and futile to attempt to?
| |
Jakob Eriksson
Sweden
| | (Moderator)
Posts 1097
02 Feb 2011 09:22
| Wouldn't the 68k be able to do that basically as fast? (And whenever, in the far future, we get subcores, just offload decompression to one of them.)
| |
JeeBee McJibble
United Kingdom
| |
Posts 85
02 Feb 2011 11:26
| Dedicated function hardware is likely smaller than a full CPU. However I believe that compressors/decompressors will require a reasonable amount of RAM to store their compressed data dictionary in - if that can be main memory then no problem, but it's a major issue if it would have to be FPGA SRAM.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
02 Feb 2011 11:51
| If i would insert a pre formated disk with files already on them would this still function?
| |
Richard Maudsley
United Kingdom
| |
Posts 821
02 Feb 2011 12:13
| If it takes load off the CPU, go for it.
| |
Morgan Johansson
Sweden
| |
Posts 99
02 Feb 2011 12:13
| Implement the promised features first (SAGA for example) and make the whole thing stable before starting to add new features, is my suggestion.
Projects tends never to get finished when features are added all the time.
| |
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
02 Feb 2011 12:33
| Morgan Johansson wrote:
|
Implement the promised features first (SAGA for example) and make the whole thing stable before starting to add new features, is my suggestion.
Projects tends never to get finished when features are added all the time.
|
Thanks for this wisdom.
| |
Wojtek P
Poland
| |
Posts 1597
02 Feb 2011 13:09
| Gunnar von Boehn wrote:
|
We could add some simple stream decompression to the NATAMI.
I think this could give a nice IO performance boost.
Normally programs on disk do compress fairly well.
If the stream decompression is on the fly, then it will not add any overhead.
Example:
If we put a NOVA de-compressor into the FPGA then we could decompress about 200MB/sec on the fly.
Lets say you have a MemoryCard of 16GB capacity and 20MB/sec peak transfer speed. With streaming decompression the card would feel as about a 32GB card with 40 MB/sec transfer rate.
Or a harddrive with 40 MB/sec transfer speed this could feel having 80MB/Sec transfer speed.
Of course the mileage will vary. If you use lots of uncompressed applications (normal AMIGA apps, icons etc) then doubling the speed is realistic. Of course for already heavy compressed items like mp3 or movies the gain will be much less.
|
For programs access time will dominate - no gain.
For compact flash cards it may be helpful as there is no access time.For large uncompressed files gain is large but you have to compress it at first - how.
Unless this accelerator provide compression too (even slower but still fast) - i am against it.
It's complete waste of FPGA space.
Better use it for example to add additions/multiplications to blitter operations.
| |
Wojtek P
Poland
| |
Posts 1597
02 Feb 2011 13:11
| Thierry Atheist wrote:
|
Sounds wicked awesome!!!
As long as it's not wicked awful, as microsoft's "double space" or whatever it was....
Could it be made so as to pass on compressing and decompressing already compressed things such as dms, xpk, rar, lha, zip, mp3, MPEG, powerpacker, etc., as it would be useless and futile to attempt to?
|
Well it's my point.1) Amiga programs are SMALL, compression doesn't make a difference much.
2) Amiga programs have often lots of files, disk access time dominate.
3) video/audio files are already compressed with lossy format. uncompressed video/audio doesn't pack well by zip-style packers.
4) it will make sense when compressing long streaams, but hardware compressor must be available.
| |
Christian Kummerow
Germany
| |
Posts 314
02 Feb 2011 13:51
| Wojtek P wrote:
|
Well it's my point. 1) Amiga programs are SMALL, compression doesn't make a difference much.
2) Amiga programs have often lots of files, disk access time dominate.
3) video/audio files are already compressed with lossy format. uncompressed video/audio doesn't pack well by zip-style packers.
4) it will make sense when compressing long streaams, but hardware compressor must be available.
|
Right, and why Compress something if you already can attach a 2TB Harddisk?
The Bigest stuff on my Harddisks is Video and it cant compressed anymore to save 50%. After it comes JPEGs for this ones its the same. Then MP3, same for this. And after it, the smallest space on HD is OS and Programs.
And i dont would use it because i cant access it with a other Amiga.
Whats if the Natami is defective, no access to my Data anymore. A Video Stream Decompression(H.263/H.264) is usefull.
| |
Matt Hey
USA
| |
Posts 733
02 Feb 2011 15:52
| Christian Kummerow wrote:
|
A Video Stream Decompression(H.263/H.264) is usefull.
|
Yes, hardware YUV (YCbCr) to RGB conversion like most gfx cards have would be good. Most have texture compression also. A modern game at high resolutions using 32 bit truecolor might use textures of 512x512, 1024x1024 or more plus mipmaps and all of a sudden the Natami doesn't have so much chip ram. Natami doesn't have a slow gfx bus to go through so compression isn't as important. The best strategy for high resolution is probably to use 16 bit and pre-dither with floyd-steinberg. I think this is good enough for now really. S3TC requires a license. The old FXT1 is free but might require some rework. It doesn't look as good as S3TC most of the time because it's more compressed (1/2 the size generally). The whole idea of texture compression is that higher resolution images with texture compression look better than lower resolution images without but are of similar size. Here is a comparison of 16 bit floyd-steinberg, S3TC, and FXT1...
EXTERNAL LINK
| |
Louis Dias
USA
| |
Posts 217
02 Feb 2011 17:09
| The team just got the board...
Let's get the original feature list up and running first the decide what to enhance with the extra space knowing exactly what we got left.Personally, storage space and thru-put is not an issue for a 100-133Mhz processor. I'd much rather enhance sprites. ;)
| |
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
02 Feb 2011 18:09
| Christian Kummerow wrote:
|
Wojtek P wrote:
|
Well it's my point.
1) Amiga programs are SMALL, compression doesn't make a difference much.
2) Amiga programs have often lots of files, disk access time dominate.
3) video/audio files are already compressed with lossy format. uncompressed video/audio doesn't pack well by zip-style packers.
4) it will make sense when compressing long streaams, but hardware compressor must be available.
|
Right, and why Compress something if you already can attach a 2TB Harddisk?
The Bigest stuff on my Harddisks is Video and it cant compressed anymore to save 50%. After it comes JPEGs for this ones its the same. Then MP3, same for this. And after it, the smallest space on HD is OS and Programs.
And i dont would use it because i cant access it with a other Amiga.
Whats if the Natami is defective, no access to my Data anymore.
A Video Stream Decompression(H.263/H.264) is usefull.
|
Erm, no offence but I get the impression that NATAMI is not the right thing for you anyway.
If your focus are compressed H264 videos get a PC and don't bother with AMIGA.
| |
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
02 Feb 2011 18:12
| Gunnar von Boehn wrote:
|
Morgan Johansson wrote:
|
Implement the promised features first (SAGA for example) and make the whole thing stable before starting to add new features, is my suggestion.
Projects tends never to get finished when features are added all the time.
|
Thanks for this wisdom.
|
No offence, but your comment was a bit unqualified, wasn't it? Do you know whether we have the decompressor available already from another project and if this will be just a matter of cut and paster for us? Also none of you knows whether the decompressor needs significant space or not. So any comments about "Save the FPGA space for Sprites etc" are not the qualified also.
| |
Wojtek P
Poland
| |
Posts 1597
02 Feb 2011 18:20
| Matt Hey wrote:
|
Christian Kummerow wrote:
|
A Video Stream Decompression(H.263/H.264) is usefull.
|
Yes, hardware YUV (YCbCr) to RGB conversion like most gfx cards have would be good.
|
Not conversion, just display mode.
| |
JeeBee McJibble
United Kingdom
| |
Posts 85
02 Feb 2011 18:24
| Do you already have such compression hardware available for incorporation? You're asking questions but not giving us details that will help us to assess whether it is a good idea or not! Does it require significant space? I mean, you could take the freely available VP8/WebM codec from Google and incorporate that! EXTERNAL LINK Video Decoder: 384 kGates logic, 52 kBytes single-port SRAM.
Can also accelerate WebP image format.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
02 Feb 2011 18:25
| gunnar my question still hasn't been replied to.
How would this feature effect a Amiga disk?(pre formated and already containing files)
| |
Matt Hey
USA
| |
Posts 733
02 Feb 2011 18:29
| @Gunnar
For no offense, your last few posts have been kind of harsh. We all have are wish list of features we want including many the PC already has because they usually make sense. Besides Thierry, most of us recognize that we will miss many of those features in early versions of Natami which is fine. We all want the Natami to be cheap and come out quickly too. Many of us don't understand the costs or priorities either but have you received any good ideas from us?
| |
Louis Dias
USA
| |
Posts 217
02 Feb 2011 18:41
| Matt Hey wrote:
|
@Gunnar
For no offense, your last few posts have been kind of harsh. We all have are wish list of features we want including many the PC already has because they usually make sense. Besides Thierry, most of us recognize that we will miss many of those features in early versions of Natami which is fine. We all want the Natami to be cheap and come out quickly too. Many of us don't understand the costs or priorities either but have you received any good ideas from us?
|
+1
We are told that something as old as and simple NEO GEO spite capabilities is not possible but somehow decoding streaming video is...
If hardware sprites were so complex, why does Amiga have the worst sprite capabilities while other lesser consoles can run circles around it? Meanwhile it can compress and decompress HD data on the fly... Is not the games market the target or are we trying to make streaming media players to be used as set-top boxes?
| |
|
Do you have ideas and feature wishes? Post them here and discuss your ideas.