Revisiting the need for additional program banks

[amit]

May, I propose a simple system, that extends upon existing feature set.

Allow disk based Exi banks. (from a system folder or a bank folder on internal HDD), it should be very easy to do for exi's.

You can currently preview any Exi directly from disk without loading it, ( I suppose the program gets copied to an edit buffer).

just enhance this feature so you can directly load the program from disk, i/e without writing to internal memory, the engine should be able to reference the source program directly (pointer) in it's edit buffer.

Since you can only have at most 16 programs at any given times, have 16 internal program edit buffers

so an entire combi, or song can be loaded from disk from different disk banks. or even virtual banks,

That gives unlimited (as in unlimited number of disk banks or just maybe 99999 banks) access to all engines other than the HD-1. Thus HD-1 can use resources freed up by programs of these engines.

if something similar could be done for HD-1 that would be great , but that may not be feasible due to sample depencencies.

[timg11]

amit,

That is a good idea, but would not be compatible with Combis and with MIDI. The reason we are "stuck" with the inflexible bank structure is because the program/bank system is defined by MIDI (and thus reflected in the Combi and the way the Combi references programs).

However if "disk based banks" are possible, it would be possible to make that work within the existing structure.

One way to do that is to have a "mapping page" in Global. It would have a table of extended banks, and allow a specific bank in a PCG file on disk to be linked to that specific MIDI bank number. It opens up possibilities for confusion if the files are changed or edited, but I think this would be more of an "experts only" feature. In theory, the disk-based banks would only be accessed if a combi that referenced them was selected. But all the existing banks could be done that way, and they apparently are not, so perhaps there is a reason it is difficult.

[Sharp]

Food for thought.

Why are the GM Banks protected memory? We can overwrite every single factory sound in sight, except for the GM Bank.

On the Trinity the GM Bank was a PCG file you loaded if you wanted it. When the Triton came out, GM Sounds became the only write protected sounds on a KORG keyboard. Not that doing that stopped people like "Triton Fun" who wrote a program that allowed you to overwrite the GM Bank on a Triton.

Regards
Sharp

[michelkeijzers]

You might well be happy to give up sample RAM for Program banks, but others might not be - especially if it means that difference between an old set of sounds fitting, and not fitting.

Personally I think a lot of people would give up like a few 100 MB for another bunch of program and/or combi banks. Actually, I would go even further, like making it possible to have 256 (or as much as possible) program and/or combi banks. The PCG file structure already supports this (hence the Virtual PCG file used by PCG Tools which has 64 extra program and combi banks).

People will possibly need to reduce the sample usage by 100 MB which is a one time step.

By a very rough calculation, 256 sets of Program, Combi, Wave Sequence, and Drum Kit banks would be something upwards of 750 MB. That's a lot of samples, even with VMT.

While you may think that people would be happy with some amount of reduction in sample RAM, I can ASSURE you that people would scream bloody murder if they had sounds that fit into memory in version X, which then did not fit in version X+1. It's a non-starter.

As noted previously, I agree that it would be nice if the memory management was dynamic. It's not - and some of that is related to how the KRONOS can still do things that native computer systems cannot, including managing the polyphony of a diverse set of synthesis algorithms and pushing the hardware right to the edge without glitching. Maybe we'll be able to make this different in the future, but if so it would be a very big change.

I think currently all samples from EXs are loaded. It would be better if only those samples are loaded which are actually used by programs.

Of course it would, and we've had the same opinion since about 2001 when we first started work on the OASYS platform. Most things are easier to say than to do, especially with an installed user-base and/or code-base. We've actually taken steps towards this in more recent updates, with the "Load required samples" feature. I agree that it would be nice to take this one step further and do it automatically.

Of course, if you're loading an EXs option bank, all samples will already be used by Programs - we don't include the samples otherwise. It's only if you're loading only a subset of the Programs that this becomes an issue.

- Dan

Thank you very much for this elaborate explanation.
Some remarks:
- I think the most wanted free slots are needed for programs, and in less amount combi banks. So 750 MB is maybe high, but of course everything cost money and is reduced from sample memory.
- I full understand that people don't like samples suddenly not to be loaded anymore, especially if they could load them before. So in this case, it is not a good idea to reduce the sample memory (they will possibly indeed scream bloody murder).
- I also understand it is quite a hard job to add flexibility in software, especially with the real time processing constraints for the Kronos engines/polyphony etc.
- The sample banks loading is indeed a very good step up. And you are right that most samples are used in most programs/combis already. However, I think it would remove some unused programs/combis if that would mean I can unload some samples (not entire sample banks) and have more space for programs.

But indeed, it seems there is not that much request for additional program banks.

So the solution stays to either use multiple PCG files or remove unwanted programs.

[amit]

amit,

That is a good idea, but would not be compatible with Combis and with MIDI. The reason we are "stuck" with the inflexible bank structure is because the program/bank system is defined by MIDI (and thus reflected in the Combi and the way the Combi references programs).

However if "disk based banks" are possible, it would be possible to make that work within the existing structure.

One way to do that is to have a "mapping page" in Global. It would have a table of extended banks, and allow a specific bank in a PCG file on disk to be linked to that specific MIDI bank number. It opens up possibilities for confusion if the files are changed or edited, but I think this would be more of an "experts only" feature. In theory, the disk-based banks would only be accessed if a combi that referenced them was selected. But all the existing banks could be done that way, and they apparently are not, so perhaps there is a reason it is difficult.

Interesting, but I think even within midi restriction, we should be able to have 128 banks with 128 programs each.

number of program banks currently being used are no more than 30 is my guess.

A more advanced system could be developed using nrpn parameters if need be.

If korg can offer than in a software upgrade it would be great, as it won't /shouldn't affect the internal sample memory

Create a FileSystem library with 128 banks of 128 patches each.
reserve first 30-32 banks for/as internal memory which link to actual memory.
leave the rest of the banks for users. (that should account for around and above, 10,000 extra program slots). SSD are fast so loading should not at all be a problem, you can already open pcg in disk mode and preview its sound.

(this is just taking into account GS standard, if you were to adapt GM2 as standard, then you could have 16384 banks.) Korg may have it's own implementation, but I doubt they would be limited to a much lesser amount in this day and time.

Standard Bank Access Midi controls, CC#0 and CC#32 followed by PC Message, should allow you to load any of 2080768 programs. , Assuming each program is 10KB, this amounts for 21.6 GB of disk space.

So if Korg can give even 1/10th of this, that is what , 2.5 GB of SSD space I don't think anyone would have problems giving up a little of it. (even one 100th would be a lot).

[Pedja]

Great discussion.
Thanks guys for this!!!

[michelkeijzers]

amit,

That is a good idea, but would not be compatible with Combis and with MIDI. The reason we are "stuck" with the inflexible bank structure is because the program/bank system is defined by MIDI (and thus reflected in the Combi and the way the Combi references programs).

However if "disk based banks" are possible, it would be possible to make that work within the existing structure.

One way to do that is to have a "mapping page" in Global. It would have a table of extended banks, and allow a specific bank in a PCG file on disk to be linked to that specific MIDI bank number. It opens up possibilities for confusion if the files are changed or edited, but I think this would be more of an "experts only" feature. In theory, the disk-based banks would only be accessed if a combi that referenced them was selected. But all the existing banks could be done that way, and they apparently are not, so perhaps there is a reason it is difficult.

Interesting, but I think even within midi restriction, we should be able to have 128 banks with 128 programs each.

number of program banks currently being used are no more than 30 is my guess.

A more advanced system could be developed using nrpn parameters if need be.

If korg can offer than in a software upgrade it would be great, as it won't /shouldn't affect the internal sample memory

Create a FileSystem library with 128 banks of 128 patches each.
reserve first 30-32 banks for/as internal memory which link to actual memory.
leave the rest of the banks for users. (that should account for around and above, 10,000 extra program slots). SSD are fast so loading should not at all be a problem, you can already open pcg in disk mode and preview its sound.

(this is just taking into account GS standard, if you were to adapt GM2 as standard, then you could have 16384 banks.) Korg may have it's own implementation, but I doubt they would be limited to a much lesser amount in this day and time.

Standard Bank Access Midi controls, CC#0 and CC#32 followed by PC Message, should allow you to load any of 2080768 programs. , Assuming each program is 10KB, this amounts for 21.6 GB of disk space.

So if Korg can give even 1/10th of this, that is what , 2.5 GB of SSD space I don't think anyone would have problems giving up a little of it. (even one 100th would be a lot).

I doubt it will be as simple as that. Even reading from the SSD cost time, and above all, processing time. The software needs to be changed quite a bit for this, and what about samples to be loaded which are part of the SSD PCG files?

With some performance penalty (meaning the need to load samples) it could be used as 'authoring' method where it is not a problem to wait a few seconds (max?) before being able to hear sounds.

[NuSkoolTone]

Food for thought.

Why are the GM Banks protected memory? We can overwrite every single factory sound in sight, except for the GM Bank.

On the Trinity the GM Bank was a PCG file you loaded if you wanted it. When the Triton came out, GM Sounds became the only write protected sounds on a KORG keyboard. Not that doing that stopped people like "Triton Fun" who wrote a program that allowed you to overwrite the GM Bank on a Triton.

Regards
Sharp

THIS. Aren't there like 8 banks of GM sounds or something?
I would LOVE to have access to overwriting these banks with something more useful!

[amit]

amit,

That is a good idea, but would not be compatible with Combis and with MIDI. The reason we are "stuck" with the inflexible bank structure is because the program/bank system is defined by MIDI (and thus reflected in the Combi and the way the Combi references programs).

However if "disk based banks" are possible, it would be possible to make that work within the existing structure.

One way to do that is to have a "mapping page" in Global. It would have a table of extended banks, and allow a specific bank in a PCG file on disk to be linked to that specific MIDI bank number. It opens up possibilities for confusion if the files are changed or edited, but I think this would be more of an "experts only" feature. In theory, the disk-based banks would only be accessed if a combi that referenced them was selected. But all the existing banks could be done that way, and they apparently are not, so perhaps there is a reason it is difficult.

Interesting, but I think even within midi restriction, we should be able to have 128 banks with 128 programs each.

number of program banks currently being used are no more than 30 is my guess.

A more advanced system could be developed using nrpn parameters if need be.

If korg can offer than in a software upgrade it would be great, as it won't /shouldn't affect the internal sample memory

Create a FileSystem library with 128 banks of 128 patches each.
reserve first 30-32 banks for/as internal memory which link to actual memory.
leave the rest of the banks for users. (that should account for around and above, 10,000 extra program slots). SSD are fast so loading should not at all be a problem, you can already open pcg in disk mode and preview its sound.

(this is just taking into account GS standard, if you were to adapt GM2 as standard, then you could have 16384 banks.) Korg may have it's own implementation, but I doubt they would be limited to a much lesser amount in this day and time.

Standard Bank Access Midi controls, CC#0 and CC#32 followed by PC Message, should allow you to load any of 2080768 programs. , Assuming each program is 10KB, this amounts for 21.6 GB of disk space.

So if Korg can give even 1/10th of this, that is what , 2.5 GB of SSD space I don't think anyone would have problems giving up a little of it. (even one 100th would be a lot).

I doubt it will be as simple as that. Even reading from the SSD cost time, and above all, processing time. The software needs to be changed quite a bit for this, and what about samples to be loaded which are part of the SSD PCG files?

With some performance penalty (meaning the need to load samples) it could be used as 'authoring' method where it is not a problem to wait a few seconds (max?) before being able to hear sounds.

Yes, it may be complex if you bring in Hd-1 engine (that uses samples etc), however for all exi, this should be no issue at all. It's already programmed to an extent that you can listen to the program wihout even loading it, if that functionality is increased to extent that the program comes into the edit buffer and writes back to file system slot (predefined slots, so that their address are known to system) instead of the internal bank slot then you are more than 95% done.

[EnSoNiQuEs~]

Food for thought.

Why are the GM Banks protected memory? We can overwrite every single factory sound in sight, except for the GM Bank.

On the Trinity the GM Bank was a PCG file you loaded if you wanted it. When the Triton came out, GM Sounds became the only write protected sounds on a KORG keyboard. Not that doing that stopped people like "Triton Fun" who wrote a program that allowed you to overwrite the GM Bank on a Triton.

Regards
Sharp

THIS. Aren't there like 8 banks of GM sounds or something?
I would LOVE to have access to overwriting these banks with something more useful!

That is (8?) more banks of Program slots that could be used. I'd give up GM sounds in a heartbeat!!!

+1

[danatkorg]

Brainstorming can be fun. Within Korg, however, we've already talked about all of these ideas long ago. Many, if not all, have also been discussed on these forums. I'll try to give a few brief responses to specific approaches.

[danatkorg]

Food for thought.

Why are the GM Banks protected memory? We can overwrite every single factory sound in sight, except for the GM Bank.

On the Trinity the GM Bank was a PCG file you loaded if you wanted it. When the Triton came out, GM Sounds became the only write protected sounds on a KORG keyboard. Not that doing that stopped people like "Triton Fun" who wrote a program that allowed you to overwrite the GM Bank on a Triton.

Regards
Sharp

THIS. Aren't there like 8 banks of GM sounds or something?
I would LOVE to have access to overwriting these banks with something more useful!

As noted in the docs, there are 256 GM Programs, plus nine GM Drum Programs. These are spread out over a number of bank/number locations, using the GM "variation" scheme, but the storage of these is per-Program, not per-Bank. So, essentially, there are about 2 banks of GM Programs.

When my group has discussed this in the past (starting with the OASYS, I believe) we were told that to be GM-compliant, the system needs to respond instantly to GM Program Change commands. So, the GM Programs need to be there.

It's also worth noting that the GM Programs are in fact used in many Combinations, so you may well use them more than you think!

[EvilDragon]

Maybe we'll be able to make this different in the future, but if so it would be a very big change.

Yes. 64-bit OS and support more than 4 GB of RAM. That's a given for the future of Kronos IMHO (the sooner, the better). It would allow much more samples to be loaded, and much more patch banks too.

[danatkorg]

I doubt it will be as simple as that. Even reading from the SSD cost time, and above all, processing time. The software needs to be changed quite a bit for this, and what about samples to be loaded which are part of the SSD PCG files?

Michel is correct here.

With some performance penalty (meaning the need to load samples) it could be used as 'authoring' method where it is not a problem to wait a few seconds (max?) before being able to hear sounds.

Yes, it may be complex if you bring in Hd-1 engine (that uses samples etc), however for all exi, this should be no issue at all.

"All EXi?" Actually, some EXi, such as the MOD-7, STR-1, and SGX-2, do indeed use sample data.

In general, the idea of loading on demand is certainly attractive. Naturally, we've discussed this internally over the years. It carries some drawbacks, including potentially long load times. Various workarounds for that might be possible. Data dependencies are also an issue, with Combinations referencing Programs, KARMA GEs, and Drum Patterns, Programs referencing Multisamples, Wave Sequences and Drum Kits (as well as GEs and Drum Patterns, when in Program mode), and Wave Sequences and Drum Kits referencing sample data. Suffice it to say that it's a tremendously complicated issue.

[danatkorg]

Interesting, but I think even within midi restriction, we should be able to have 128 banks with 128 programs each.

and

(this is just taking into account GS standard, if you were to adapt GM2 as standard, then you could have 16384 banks.) Korg may have it's own implementation, but I doubt they would be limited to a much lesser amount in this day and time.

Actually, MIDI - and the KRONOS - already uses Bank Select for 128 x 128 banks (16,384), with CCs 0 and 32. This has nothing to do with GM or GS. Korg may have been the first to adopt Bank Select, back in the Wavestation days.

- Dan