| Stream Decompression | page 1 2 3 4 5 6 7 8
|
|---|
|
|---|
SID Hervé
France
| |
Posts 663
11 May 2011 13:38
| Gunnar von Boehn wrote:
|
We could add some simple stream decompression to the NATAMI.
I think this could give a nice IO performance boost.
Normally programs on disk do compress fairly well.
If the stream decompression is on the fly, then it will not add any overhead.
|
Is it canceled?
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
11 May 2011 18:36
| SID what is important is that the format used to compress is available on a classic for obvious reasons.
What format to use none can agree to so, yes it seem that way to me.
| |
SID Hervé
France
| |
Posts 663
11 May 2011 20:49
| I quote: "If the stream is on the fly decompression, then it will not add any overhead". What matters is the principle itself. The question of the format is secondary and, if consensus is not obtained, I think its choice and its integration belongs to the development team.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
12 May 2011 02:34
| SID your right but if I cannot read out my data after a hardware failure I for one am 'not a happy camper'.
Plain and simple it has to be a standard format that even Thierry should be able to use without too much trouble.(I know i am asking for a lot here)
| |
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
12 May 2011 05:30
| Marcel Verdaasdonk wrote:
|
SID your right but if I cannot read out my data after a hardware failure I for one am 'not a happy camper'.
Plain and simple it has to be a standard format that even Thierry should be able to use without too much trouble.(I know i am asking for a lot here)
|
I don't get your post. You are stating the obvious and using it as argument for something totally unrelated.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
12 May 2011 06:04
| What i mean with it is that i would like to operate the disk in a other machine outside the natami.
| |
Gunnar von Boehn
Germany
| | (Moderator)
Posts 5775
12 May 2011 07:07
| Marcel Verdaasdonk wrote:
|
What i mean with it is that i would like to operate the disk in a other machine outside the natami.
|
I see so you misunderstood the discussion here.
I suppose you never used an AMIGA which had a filesystem compression handler before? Otherwise this would certainly have been more clear to you. To me your post always sounded like
"But I want WATER TO BE WET (I know I ask for a lot here)"
Which made me wonder what you talk about...
| |
SID Hervé
France
| |
Posts 663
12 May 2011 17:10
| Marcel Verdaasdonk wrote:
|
SID your right but if I cannot read out my data after a hardware failure
|
Do you remember, you have recommended to do a safety backup?On the other hand, it might seem wise to offer this feature as optional. And anyway, my original question concerned the status of this proposal, so has it been abandoned or asleep?
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
12 May 2011 18:13
| Gunnar von Boehn wrote:
|
"But I want WATER TO BE WET (I know I ask for a lot here)"
|
Gunnar your testing me right?(I can make such a bad joke here) Anyhow let's wake Cinderella up and face the music.
Fact. Classics do not get any younger.
Fact. Plenty of alternative systems that support IDE
Fact. SCSI was the defacto standard in the Amiga hayday. Let's see if i can get you to see it my way in this case.
System one a standard A600, system two a slot 1 pentium II with EIDE support.(better known as ATA3)
Can i open the A600's drive on the PC?
Can i open the PC's drive on the A600? I can tell you this, on a single drive i can have partition for both, but only the Amiga can be aware of them both.(this is a fact Rigid Disk Block can be in the first 16 sectors the MBR needs to be in the first.)
Let's make it more fun PC with Linux and Amiga FFS support. you see where this is heading?
I know, I know and Even Dos had a fast way to lose your data. ;) (compression was and still is a dumb thing to do when you need to recover data, DOS and NTFS taught me this much, catweasel taught me AFS is redundant to the point that it's plain stupid.)
| |
SID Hervé
France
| |
Posts 663
02 Mar 2012 18:36
| Gunnar von Boehn wrote:
|
We could add some simple stream decompression to the NATAMI.
I think this could give a nice IO performance boost.
Normally programs on disk do compress fairly well.
If the stream decompression is on the fly, then it will not add any overhead.
Example:
If we put a NOVA de-compressor into the FPGA then we could decompress about 200MB/sec on the fly.
Lets say you have a MemoryCard of 16GB capacity and 20MB/sec peak transfer speed. With streaming decompression the card would feel as about a 32GB card with 40 MB/sec transfer rate.
Or a harddrive with 40 MB/sec transfer speed this could feel having 80MB/Sec transfer speed.
Of course the mileage will vary. If you use lots of uncompressed applications (normal AMIGA apps, icons etc) then doubling the speed is realistic. Of course for already heavy compressed items like mp3 or movies the gain will be much less.
|
Hello,
Sorry to revive this option, but does this idea will be present?
Thank you
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
02 Mar 2012 22:20
| SID why do you need this to be in hardware?
| |
Jakob Eriksson
Sweden
| | (Moderator)
Posts 1097
02 Mar 2012 22:55
| Even if we decide that this decompression feature is desirable, it should be prototyped in software first. So don't hold your breath. :-)
| |
SID Hervé
France
| |
Posts 663
03 Mar 2012 01:18
| Marcel Verdaasdonk wrote:
|
SID why do you need this to be in hardware?
|
May be because the OS could not have to supervise this kind of work.Jakob Eriksson wrote:
|
Even if we decide that this decompression feature is desirable, it should be prototyped in software first. So don't hold your breath. :-)
|
Obviously this is not a priority. But, anyway, it remains a very interesting idea if it is integrated in hardware.Thank you for caring about my health, it's nice of you.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
03 Mar 2012 08:55
| It's a interesting method of creating a vendor lock-in.
| |
SID Hervé
France
| |
Posts 663
03 Mar 2012 10:13
| Yes, you're right. Proposed an option has always been an interesting method to the creation of a lock. It is best to impose it, this will allow freely choose.
| |
Marcel Verdaasdonk
Netherlands
| |
Posts 3976
03 Mar 2012 10:33
| SID Hervé wrote:
|
Yes, you're right. Proposed an option has always been an interesting method to the creation of a lock. It is best to impose it, this will allow freely choose.
|
Pardon? Stream compression and decompression create such a lock-in not remove them! If you don't understand such a simple fact you should read up on it.
Depending on the method of compression and the ability of turning it on and off on the fly would only make this worse! This would not work around the FS limit since it is not intended to do this.
It makes it very hard for one to put software on a drive in a PC and then mount the drive in the Natami, because Hardware expects a certain format!
| |
Thomas Hirsch
Germany
| | (MX-Board Owner)
Posts 647
03 Mar 2012 13:23
| Actually this is the way to go. The current demands of multimedia can at some point be broken down to some kind of streaming. Natami unlimited support of RAW CPU power, but in regards of memory throughput it is quite state of the art. This means if we want to try the classical "PC approach" in decoding data streams by the CPU we will be not be better that any other hobby project. Three or five years I bought a DVD player for about 25Euro. - Then, shouldn't it be possible for a today's 150Euro FPGA to achieve the same and not even getting warm by that? - Of course the Processor will be the overseeing master. But the data movement, copying, calculation and manipulation should be done where the data is: in an DMA core in the FPGA. So, if there are people out there who have deeper knowledge about codecs and what data transformations could be handy - I/we would appreciate input in that regard in the near future.
| |
Jens Drößler
Germany
| |
Posts 136
03 Mar 2012 14:28
| I think this is not a bad idea. But how about not attaching the hardware decompressor not directly to the ATA port (which would only make it useable for that port, not for SATA PCI cards, not for USB...) but have it as an additional "processor" with only that function? You could code an XPK module which will use that processor if available instead of using the CPU (and if not available it will do it on the CPU). That way, we would have hardware acceleration on Natami, but still simply could attach the same disk to any other Amiga with IDE. Also, the decompressor wouldn't use FPGA space for every port. Maybe this way there could be two or three different decompressors for different data types, or a hardware compressor. Also, by using XPK we could easily select ourselves what to compress and what not. I know, this wouldn't be as effective as doing it before the data even DMA into RAM, but it has its avantages, I think. If there is enough FPGA space left in the end, maybe the compression would not be hardcoded, but use a "compression/decompression-optimized" DSP to load one of several compression algorithms.
| |
Jakob Eriksson
Sweden
| | (Moderator)
Posts 1097
03 Mar 2012 15:39
| If we had DSP or SIMD-like extra cores, they could do stuff not possible with normal PC architecture. For instance it could stream audio data directly from HDD to the sound port, without even touching the RAM bus. It could be a little like when CDTV games could play CD music in games by mixing the analog signal in with the Amiga sound, but this would be digital instead.
| |
SID Hervé
France
| |
Posts 663
03 Mar 2012 18:37
| So the idea was not abandoned or retained. It just requires prior consideration. This is a good news. The initial idea concerned only a general compression. Is it really necessary to opt for a compression based on data type? Thank you.
| |
|
Do you have ideas and feature wishes? Post them here and discuss your ideas.