 |
Korg Forums
A forum for Korg product users and musicians around the world.
Moderated Independently.
Owned by Irish Acts Recording Studio & hosted by KORG USA
|
| View previous topic :: View next topic |
| Author |
Message |
pandel
Junior Member
Joined: 21 Jul 2006
Posts: 87
Location: Germany, Moers
|
 Posted: Tue Feb 16, 2010 3:21 pm Post subject: Any docs about the PCG file format? |
 |
|
Hello everybody!
Haven't been here for a while... but now I'm looking for some kind of documentation about how data is arranged in a PCG file.
I'd like to try to develop some kind of library software by myself and the neccessary file format documentation seems to exist, because some people out here have done so before.
Does anybody know (perhaps especially Daz or Dan), if there is a way for me to get some more descriptive documention that is needed for such a project?
Regards,
Holger |
|
| Back to top |
|
 |
danatkorg
Product Manager, Korg R&D
Joined: 21 Jan 2005
Posts: 4204
Location: California, USA
|
| Posted: Tue Feb 16, 2010 11:20 pm Post subject: Re: Any docs about the PCG file format? |
|
|
| pandel wrote: | Hello everybody!
Haven't been here for a while... but now I'm looking for some kind of documentation about how data is arranged in a PCG file.
I'd like to try to develop some kind of library software by myself and the neccessary file format documentation seems to exist, because some people out here have done so before.
Does anybody know (perhaps especially Daz or Dan), if there is a way for me to get some more descriptive documention that is needed for such a project?
Regards,
Holger |
I'm sorry, but there is no documentation for the format. As far as I'm aware, any apps which access or create PCG files (such as those that Daz hass created) have used the system exclusive format as a guide. Unfortunately, as of software version 1.3, the PCG format has diverged somewhat from the system exclusive format. I wish that I could have a better answer for you.
_________________
Dan Phillips
Manager of Product Development, Korg R&D
Personal website: www.danphillips.com
For technical support, please contact your Korg Distributor: http://www.korg.co.jp/English/Distributors/
Regretfully, I cannot offer technical support directly.
If you need to contact me for purposes other than technical support, please do not send PMs; instead, send email to dan@korgrd.com |
|
| Back to top |
|
|
Charlie
Platinum Member
Joined: 30 Oct 2006
Posts: 997
Location: Austria
|
|
| Back to top |
|
|
X-Trade
Moderator
Joined: 14 Feb 2006
Posts: 6494
Location: Leeds, UK
|
| Posted: Wed Feb 17, 2010 10:03 am Post subject: |
|
|
| Charlie wrote: | | What does "diverged" mean? |
It means split away, separated out, parted...
For example at points on a railway track, the two tracks diverge.
I think Dan is saying that, since OS 1.3, the PCG format is no longer strictly a sysex based file - it now contains bits of data in other formats too, for whatever reason.
_________________
Current Gear: Kronos 61, RADIAS-R, Volca Bass, ESX-1, microKorg, MS2000B, R3, Kaossilator Pro +, MiniKP, AX3000B, nanoKontrol, nanoPad MK II,
Other Mfgrs: Moog Sub37, Roland Boutique JX03, Novation MiniNova, Akai APC40, MOTU MIDI TimePiece 2, ART Pro VLA, Focusrite Saffire Pro 40.
Past Gear: Korg Karma, TR61, Poly800, EA-1, ER-1, ES-1, Kawai K1, Novation ReMote37SL, Boss GT-6B
Software: NI Komplete 10 Ultimate, Arturia V Collection, Ableton Live 9. Apple OSX El Capitan on 15" MacBook Pro |
|
| Back to top |
|
|
pandel
Junior Member
Joined: 21 Jul 2006
Posts: 87
Location: Germany, Moers
|
| Posted: Wed Feb 17, 2010 11:15 am Post subject: Re: Any docs about the PCG file format? |
|
|
| danatkorg wrote: |
I'm sorry, but there is no documentation for the format. |
Does that really mean "no documentation" at all or only nothing publicly available? Could you explain why Korg keeps it as a big secret? I think now that there'll be no more active development for the OASYS, Korg should be happy to see that there is some kind of active community development effort going on! With its outstanding capabilities and the myriads of possibilities the OASYS has to offer, a much smarter way of organizing progs and combis is needed.
I think, even if one would have to sign some kind of NDA or so, Korg shouldn't stop those efforts...
@Edit:
Is any detailed sysex/midi implementation documentation available?
_________________
Holger |
|
| Back to top |
|
|
Daz
Retired
Joined: 01 Jan 2002
Posts: 10829
|
| Posted: Wed Feb 17, 2010 3:38 pm Post subject: |
|
|
If you only wish to work with PCG files in a simple way (moving programs or combis around for example) then you only need to know one thing. The PCG file uses a very RIFF chunk like format that you'll have seen used by MIDI, WAV, BMP files etc. If you look in your Parameter Guide you'll see the specification is given for KSF files. KSF files are also RIFF like, and it will give you a few clues as to how to work with PCG files which work similarly. You don't really need a document to work it out, unless you are looking at parameter details for individual programs/combis rather than just looking at groups of those.
I don't think Korg are trying to keep this stuff secret, it's more likely just a matter of resources. Should your stretched development team be working on developing new products or preparing documentation for third party developers ? The former right ? I wish there were more open documentation and I have got stuck with some of my own developments, because I don't have the time to chase or decode details. I have my actual work to do !  I do understand where Korg are coming from though, because I appreciate from personal experience the extra effort that is required to maintain high quality developer focused documentation.
Daz. |
|
| Back to top |
|
|
pegnafroy
Full Member
Joined: 19 Apr 2007
Posts: 201
Location: Penaflor, Santiago of Chile
|
| Posted: Wed Feb 17, 2010 3:45 pm Post subject: |
|
|
And keep in mind that Dan Phillips is who wrote the most part of the manual, so the docu team is very, very narrow.
_________________
http://www.490.cl/votar.php?v=85
Si no puedes contra ellos, cambia de vocalista... |
|
| Back to top |
|
|
pandel
Junior Member
Joined: 21 Jul 2006
Posts: 87
Location: Germany, Moers
|
| Posted: Wed Feb 17, 2010 4:14 pm Post subject: |
|
|
Many thanks, Daz! Regarding the RIFF like structure, I'll look into it.
| Quote: | | I don't think Korg are trying to keep this stuff secret, it's more likely just a matter of resources. ... Wink I do understand where Korg are coming from though, because I appreciate from personal experience the extra effort that is required to maintain high quality developer focused documentation. |
I absolutely understand and agree in general! And I would not expect them to prepare those documents now. I rather thought, they would exist somehow nearly as long as the OASYS exists. At least some basics, like the complete MIDI implementation or things like a file structure. Imagine the developer crew, which knows all the magic details, getting into severe trouble after having a nasty mushroom soup for diner before the OASYS was discontinued... 
@pegnafroy
Yep, I know that  ! I very much appreciate his work on the documentation!
But never mind! I understand that I have to investigate things by myself, even though I have to admit, that for my part I'm not very good at analysing file structures. If I have a format description, everything's ok, but if I have to find out everything by myself, I'm always afraid of doing something terribly wrong...
@Daz
But what I don't understand is that somehow some kind of information should exist, when I understand this correctly: http://www.korgforums.com/forum/phpBB2/viewtopic.php?t=17438&start=48
Did you ever get the information you were looking for?
_________________
Holger |
|
| Back to top |
|
|
Daz
Retired
Joined: 01 Jan 2002
Posts: 10829
|
| Posted: Thu Feb 18, 2010 1:30 am Post subject: |
|
|
Here is some C++ snippets from the code I wrote from the Oasys that might help to show how things work.
Firstly here are RIFF like chunk id's for each blob of data in the PCG :
| Code: |
// ***
// *** Oasys PCG chunk ids
// ***
typedef DWORD CHKID;
#define DEFINE_CHKID(magic) (magic)
const CHKID KORG = DEFINE_CHKID('KORG');
const CHKID PCG1 = DEFINE_CHKID('PCG1');
const CHKID DIV1 = DEFINE_CHKID('DIV1');
const CHKID INI1 = DEFINE_CHKID('INI1');
const CHKID INI2 = DEFINE_CHKID('INI2');
const CHKID PRG1 = DEFINE_CHKID('PRG1');
const CHKID PBK1 = DEFINE_CHKID('PBK1');
const CHKID MBK1 = DEFINE_CHKID('MBK1');
const CHKID CMB1 = DEFINE_CHKID('CMB1');
const CHKID CBK1 = DEFINE_CHKID('CBK1');
const CHKID DKT1 = DEFINE_CHKID('DKT1');
const CHKID DBK1 = DEFINE_CHKID('DBK1');
const CHKID WSQ1 = DEFINE_CHKID('WSQ1');
const CHKID WBK1 = DEFINE_CHKID('WBK1');
const CHKID GLB1 = DEFINE_CHKID('GLB1');
|
Here are the structures for each chunk type :
| Code: |
// ***
// *** Oasys PCG File Structures
// ***
#pragma pack(1)
struct PCGKORGHeader
{
CHKID idKorg; // 'KORG'
BYTE bProductId; // 0x70
BYTE bFileType; // 0x00 = PCG
BYTE bMajVer;
BYTE bMinVer;
BYTE abPad[ 8 ];
PCGKORGHeader() { memset( this, 0, sizeof(*this) ); }
};
struct PCGChunk
{
CHKID idChunk;
DWORD cbChunk;
DWORD dwX; // not sure what this is -- entity count ?
PCGChunk() { memset( this, 0, sizeof(*this) ); }
};
struct PCGBankHeader // used for all data types
{
DWORD cItems; // number of items (progs, combis, dkits)
DWORD cbItem; // size of each item
DWORD dwBankId; // 0x00-0x04 = A-E, 0x8000 = F, 0x20000 = U-A etc.
PCGBankHeader() { memset( this, 0, sizeof(*this) ); }
};
#pragma pack()
|
Now here are the two functions to read/validate the file and then get references to all the banks in this file :
| Code: |
bool CPCGFile::Read( const char* pszFile )
{
// *** get file size
struct stat st;
if( 0 != stat( pszFile, &st ) )
{
m_strRWError = "Failed to retrieve file status";
return false;
}
size_t cbFile = st.st_size;
// *** attempt to open file
std::ifstream ifs( pszFile, std::ios::binary|std::ios::in );
if( ! ifs )
{
m_strRWError = "Failed to open file for reading";
return false;
}
// *** validate KORG and PCG1 headers
PCGKORGHeader KorgHdr;
ifs.read( (char*)&KorgHdr, sizeof(KorgHdr) );
if( twiddle( KorgHdr.idKorg ) != KORG || KorgHdr.bFileType != 0x00 || ! ifs )
{
m_strRWError = "File does not have a valid KORG/PCG header";
return false;
}
if( KorgHdr.bProductId != 0x70 ) // Oasys
{
m_strRWError = "PCG File is not in Korg Oasys format";
return false;
}
PCGChunk pcg1;
ifs.read( (char*)&pcg1, sizeof(pcg1) );
if( twiddle( pcg1.idChunk ) != PCG1 || ! ifs )
{
m_strRWError = "PCG File does not have a valid PCG1 header";
return false;
}
// *** rewind to the beginning of file and suck the whole thing into memory
ifs.seekg( 0 );
m_pbData = new BYTE[ cbFile ];
ifs.read( (char*)m_pbData, (std::streamsize)cbFile );
m_cbData = cbFile;
// *** now parse the bank data
parseBanks();
return true;
}
void CPCGFile::parseBanks()
{
size_t iPos = 0;
bool fBanksDone;
// *** jump over KORG and PCG1 header
iPos += sizeof(PCGKORGHeader) + sizeof(PCGChunk);
// *** loop round pulling in chunks/banks
while( iPos <m_cbData>idChunk ) )
{
case PRG1:
case CMB1:
case DKT1:
case WSQ1:
fBanksDone = false;
while( iPos <m_cbData>idChunk ) )
{
case PBK1:
case MBK1:
case CBK1:
case WBK1:
case DBK1:
{
iPos += sizeof(PCGChunk);
PCGBankHeader bkhdr = *(PCGBankHeader*)&m_pbData[ iPos ];
bkhdr.cbItem = twiddle( bkhdr.cbItem );
bkhdr.cItems = twiddle( bkhdr.cItems );
bkhdr.dwBankId = twiddle( bkhdr.dwBankId );
// construct CPCGBank instace with pointer to beginning of bank data, the bank type, and the bank header which
// indicates the number of items in the bank and how large each is
CPCGBank bank( &m_pbData[ iPos + sizeof(PCGBankHeader) ], twiddle(pBankChunk->idChunk), bkhdr );
m_banks.push_back( bank );
iPos += twiddle( pBankChunk->cbChunk );
}
break;
default:
fBanksDone = true;
break; // chunk is not a bank ... leave for outer loop
}
} // while( iPos < m_cbData ) ... reading banks
break;
default: // jump over chunks we don't care about
iPos += twiddle( pChunk->cbChunk );
break;
} // switch( pChunk->idChunk )
} // while( iPos < m_cbData )
}
|
So the file consists of a bunch of chunks for each kind of data, but the ones we're specifically interested in are PRG1, CMB1, DKT1 and WSQ1 which contain the Program, Combi, Drumkit and Wavesequence banks. Once you're inside each of those chunk types you'll find a number of chunks for each bank. So a PRG1 chunk contains a number of PBK1 chunks, one for each bank of Programs in the PCG file. The data in the PBK1 chunk is data for all of the programs in that bank. You can slice it up using the cbItems member of the PCGBankHeader structure, that is the size of each item.
Just for larks note that EXi programs are not stored in PBK chunks, but MBK chunks. On the Triton MBK chunks were used to store MOSS programs, hence the M in the name. Exciting trivia, eh ?
Daz. |
|
| Back to top |
|
|
Daz
Retired
Joined: 01 Jan 2002
Posts: 10829
|
| Posted: Thu Feb 18, 2010 2:09 am Post subject: |
|
|
p.s. This kind of format is useful to know about. As I say it's not just PCG files that work this way, or even just music related files like WAV and AIFF. Many image file formats are similar, such as BMP, TIFF, and even some RAW camera files IIRC.
p.p.s As you'll see no sysex information is used in the code. You only need the sysex information once you start wanting to extract parameters from the data stored in those blobs in the file. The way that program/combi/dkit/wseq parameters are stored in the PCG chunks is very closely related to the format used when sending the exact same information using MIDI/Sysex. This is true of the Triton, M3 and Oasys.
Daz. |
|
| Back to top |
|
|
Kevin Nolan
Approved Merchant
Joined: 04 Dec 2005
Posts: 2524
Location: Dublin, Ireland
|
| Posted: Thu Feb 18, 2010 10:11 am Post subject: |
|
|
Korg are are a fine company in many ways, but they do NOT understand the concept of professional support in the keyboard performance world.
They rarely offer in-depth tutorials on their products; they have a lousy service policy and infrastructure, they make absolutely no effort to publicise spare parts, they do not make service documentation available and have no comprehension of a global support network (and why so often you'll see 'special offers' from Korg US for example but with no cooperation to offer similar deals to other regions (that bugs me in particular because surely Korg should know that every offer made available to one region only constitutes a slap in the face to users in other reasons; but they don't get that)).
So while OASYS is great and Korg are an earnest company in so many ways - most especially in their commitment to new development and to keyboard players / synthesistis; they could do with dropping some of the apparent protectionist paranoia that seems to emanate from Korg Japan and that does affect their persona, but also their own capacity/desire to support their user community.
In particular, they should have a comprehensive web based spares and accessories portal (take a look at Yamaha US spares and accessories - its amazing); and they should release service documents for sale for all end-of-life instruments to service their well known flawed parts - faint lcd screens, flimsy switches, broken joy sticks and so on.
By comparison, Yamaha are awesome. Yamaha-Kemble UK are simply incredibly accommodating with the sale of accessories and service manuals and will even seek out parts in Yamaha Japan in particular if they are not available in the UK; while as I said the Yamaha US spares web portal is amazing (though I wish that was world wide too). But world wide Yamaha are great at providing service documentation and Korg should also follow suit in this regard
Kevin.
Last edited by Kevin Nolan on Thu Feb 18, 2010 10:13 am; edited 1 time in total |
|
| Back to top |
|
|
Charlie
Platinum Member
Joined: 30 Oct 2006
Posts: 997
Location: Austria
|
| Posted: Thu Feb 18, 2010 10:12 am Post subject: |
|
|
@X-Trade: thanks for explaining the word - my fault having it written like that. I know what "diverged" means in general, but I didn't understand what Dan meant regarding the change in pcg-file-structure.
_________________
www.charliemclight.com/en/home.html
www.facebook.com/charliemclight |
|
| Back to top |
|
|
pandel
Junior Member
Joined: 21 Jul 2006
Posts: 87
Location: Germany, Moers
|
| Posted: Thu Feb 18, 2010 1:58 pm Post subject: |
|
|
Need to use the 'Sina'ticons here 
Dear Daz,
what an amazing answer and, I hope you believe me, it's much more than I expected ever!!!!!
So, as soon as I've time I'll give it a try!
Many, many thanks!
_________________
Holger |
|
| Back to top |
|
|
danatkorg
Product Manager, Korg R&D
Joined: 21 Jan 2005
Posts: 4204
Location: California, USA
|
| Posted: Fri Feb 19, 2010 1:00 am Post subject: |
|
|
| Daz wrote: |
I don't think Korg are trying to keep this stuff secret, it's more likely just a matter of resources. Should your stretched development team be working on developing new products or preparing documentation for third party developers ? The former right ? |
With regards to the PCG file format, that is exactly the case. I truly wish that we had documentation for this, but we simply do not.
Best regards,
Dan
_________________
Dan Phillips
Manager of Product Development, Korg R&D
Personal website: www.danphillips.com
For technical support, please contact your Korg Distributor: http://www.korg.co.jp/English/Distributors/
Regretfully, I cannot offer technical support directly.
If you need to contact me for purposes other than technical support, please do not send PMs; instead, send email to dan@korgrd.com |
|
| Back to top |
|
|
pandel
Junior Member
Joined: 21 Jul 2006
Posts: 87
Location: Germany, Moers
|
| Posted: Fri Feb 19, 2010 10:22 am Post subject: |
|
|
Dan, as I said before, I highly appreciate your work on the documentation and absolutely believe that *you* personally would offer something if you had something.
I don't want to argue, I just thought, that the different development teams that worked on the parts of the OS would have had some basic doc stuff to coordinate what they were doing. I mean, I really don't believe the USA dev team has had to re-discover all the MIDI messages (Midiox ) introduced by the Japanese dev team to be sure not using them twice, only because the Japanese had not documented it...
And finally, I think, a product like the OASYS has to be accompanied by a complete (!) end user documentation, even if some work still has to be done! I would expect that from every high price product. (Besides, for me that includes the availability of a service manual after the product has come to an official end, too.)
Don't get me wrong! The op guide and the param guide are both exceptional and highly detailed. I love them, nothing to discuss here! And I understand the ressource problematic but it's a pity!
Have a nice weekend  !
_________________
Holger |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
