AbstractA method for electronically processing and storing musical information involves separating the musical information into a plurality of segments, each representing some portion of a measure, and assigning a sequential time dimension value to each segment. The musical information is also separated into plurality of channels, with each channel representing a sound source and having a sound dimension value assigned to it. The musical information for a given channel and segment is stored in a memory array by associating the musical information corresponding to a given channel and segment with the memory array node specified by the time dimension value and sound dimension value assigned to the given channel and segment.ClaimsI claim:
1. An apparatus for electronically storing and processing music information comprising:
means for receiving electronic signals representing a time-ordered sequence of notes and rests in musical composition to be performed by one or more sound sources;
means for identifying in said electronic signals the pitch of each note and the duration of each note and rest and means for assigning each note and rest to a first measure and succeeding sequentially ordered measures; and
means for storing an electronic representation of the pitch and duration of each note and the duration of each rest in a computer memory array comprised of a two dimensional framework of storage nodes, each storage node being associated with a unique one of said first and succeeding measures and a unique one of said one or more sound sources.
2. The apparatus as recited in claim 1 wherein the pitch of each note is expressed as a base value and a displacement from said base value.
3. The apparatus as recited in claim 1 wherein the duration of each note and each rest is expressed as one or more beat units said beat unit being a specified absolute time interval.
4. The apparatus as recited in claim 1 wherein the means for storing an electronic representation of the pitch and duration of each note and the duration of each rest is a digital computer memory having a field format that is the same for each note.
5. The apparatus as recited in claim 4 wherein sid field format comprises:
a comparator field for defining a sequential position;
a tag field for differentiating various acoustic or graphic attributes associated with a note, a rest, or a musical composition; and
a data field containing encoded data expressing one or more of said attributes or a pointer leading to one or more of said attributes.
6. The apparatus as recited in claim 1 wherein said memory array has at least one additional dimension implemented by vectors associated with one or more of said storage nodes, with said vectors containing additional electronic representations specifying graphical representations of textual, interpretive, pitch or duration attributes associated with the notes in the storage node associated with said vector.
7. The apparatus as recited in claim 1, further comprising:
means for retrieving the electronic representations of the pitch and duration of each note and the duration of each rest from said memory array;
means for translating all of the retrieved electronic representations stored in each storage node associated with a single sound source into a graphical representation of a musical staff with notes and rests; and
means for placing in the graphical representation of a musical staff, measure markers that demarcate the notes associated with the first and succeeding measures as indicated by the storage mode at which each note and rest is stored.
8. The apparatus as recited in claim 1, further comprising:
means for retrieving said electronic representations of the pitch and duration of each note and the duration of each rest from said memory array, starting with the notes in the storage node corresponding to the first measures stored in said memory array for each sound source and continuing sequentially for notes in the storage nodes corresponding to the second and succeeding measures in said memory array; and
means for translating the retrieved electronic representations into electronic output signals to at least one sound source, starting with the first note in any measure of the storage node corresponding to the first measures stored in said memory for each sound source and joining with said first note any notes to be performed at the same time, then continuing with the next succeeding note and any notes to be performed in said first measure and further continuing with the notes in measures of the storage node corresponding to the second and succeeding measures at the same time.
9. A method for electronically processing and storing musical information using a programmable data processing system, the steps comprising:
providing the programmable data processing system with a plurality of data signals representing musical information; and
using the programmable data processing system to perform the steps of:
separating the musical information into a plurality of segments, each segment representing some portion of a measure;
assigning a sequential time dimension value to each segment;
separating the musical information into a plurality of channels, each channel representing a sound source;
assigning a sound dimension value to each channel; and
storing the musical information for a given channel and segment by associating the musical information corresponding to a given channel and segment with the time dimension value and sound dimension value assigned to the given channel and segment.
10. A music processing apparatus for representing musical information comprising:
means for selectively inputting musical information;
programmable data processing means operably connected to the means for selectively inputting musical information for electronically representing, storing and retrieving the musical information in a memory means associated with the programmable data processing means for storing information; and
a multi-dimensional data structure framework within the memory means of the programmable data processing means having:
a time dimension for separating the musical information into a plurality of segments, each segment representing some portion of a measure; and
a sound dimension for separating the musical information into a plurality of channels, each channel representing a sound source, such that a plurality of framework intersection points are defined by the intersections of the time dimension and the sound dimension,
whereby the musical information for a given sound source and measure is stored at the framework intersection point corresponding to a time dimension value and a sound dimension value that correspond to the given instrument and measure.
11. The music processing apparatus of claim 10 wherein the framework further comprises a performance dimension accessible at each framework intersection point for storing additional musical information representing one or more multiple renditions of a particular measure as played by a particular instrument.
12. A method for processing musical information comprising the steps of:
inputting the musical information into a programmable data processing means for electronically representing the musical information;
separating the musical information into a plurality of measures;
assigning a time dimension value to each measure;
separating the musical information into a plurality of channels;
assigning a sound dimension value to each channel;
storing the musical information for a given channel and measure by associating the musical information corresponding to a given channel and measure with an array location specified by the time dimension value and the sound dimension value assigned to the given instrument and measure.
13. The method of claim 12 further comprising the steps of:
retrieving the musical information for a specified range of instruments and measures by:
specifying all of the combinations of time dimension values and the sound dimension values for the specified range of instruments and measures;
determining whether musical information exists for each combination of the time dimension values and sound dimension values;
retrieving the musical information for each combination of time dimension value and sound dimension value that exists; and
generating a whole rest for each combination of a time dimension value and a sound dimension value that has no associated musical information; and
outputting the retrieved musical information.
14. A method for electronically representing musical information using a programmable data processing system, the steps comprising:
providing the programmable data processing system with a plurality of data signals representing musical information; and
using the programmable data processing system to perform the steps of:
storing a plurality of entries that represent rhythmic, melodic, and interpretive aspects of the musical information, each entry comprising one unit of music data selectively representing a rest, a note or a chord and a set of details associated with the entry;
linking successive entries for a given instrument together in time sequence order;
grouping a portion of a sequence musical information to be associated with a specified measure for a given instrument by:
assigning a first pointer to the successive entries for the given instrument to designate the first entry to be included in the measure; and
assigning a last pointer to the successive entries for the given instrument to designate the last entry to be included in the measure.
15. A method for electronically representing pitch information associated with musical information using a programmable data processing system, the steps comprising:
providing the programmable data processing system with a plurality of data signals representing pitch information; and
using the programmable data processing system to perform the steps of:
defining a harmonic reference as an absolute frequency;
defining a tone center as an octave key signature relative to the harmonic reference;
defining a diatonic displacement for dividing the tone center into diatonic steps; and
representing the pitch information for a note as a relative displacement of diatonic steps from the tone center,
whereby the octave key signature associated with a given note is transposed without computation as a result of a change in the tone center.
16. A music processing apparatus for representing musical information, the musical information comprising a plurality of entries with each entry representing one unit of music data that may be a rest, a single note or a chord and a set of attributes associated with that entry, comprising:
a programmable data processing means for executing a program; and
memory means operably connected to the programmable data processing means for storing the musical information, the memory means including;
a multi-dimensional data structure framework for storing the musical information having:
a time dimension for separating the musical information into a plurality of segments, each segment representing some portion of a measure; and
a sound dimension for separating the musical information into a plurality of channels, each channel representing a sound source,
such that a plurality of framework intersection points are defined by a pair of discrete values of the time dimension and the sound dimension;
an entry pool for storing the entries that comprise the musical information; and
a plurality of frames assigned to the framework intersection points, each frame having:
a first and last pointer to the entry pool for designating entries to be included in that frame; and
a frame specification record for storing additional attributes associated with the musical information contained in the frame.
17. A music processing apparatus including programmable data processor means for executing a program and memory means for storing musical information, comprising:
a plurality of frames included in the memory means, each frame for storing musical information associated with a measure for one of a plurality of sound sources;
a plurality of assignment records included in the memory means, each assignment record indicating the measure and instrument assigned to that frame; and
means for retrieving the musical information from the memory means by searching the assignment records for sequential measures for a given sound source and retrieving the musical information stored in the corresponding frame if an assignment record is found and placing a whole rest in the measure if an assignment record is not found for the measure and sound source being searched for.
18. A common data structure for representing musical information comprising:
a music framework having a sound dimension and a time dimension:
means for storing page-related data for the music framework such that the page-related data is identified by a page number;
means for storing instrument-related data along the sound dimension of the music framework such that the instrument-related data is identified by an instrument number;
means for storing measure-related data along the time dimension of the music framework whereby the measure-related data is identified by a measure number;
means for storing frame-related data at intersection points of the sound dimension and the time dimension of the music framework such that the frame-related data is identified by a frame number;
means for storing entry-related data such that the entry-related data is identified by an entry number; and
means for relating designated portions of the entry-related data to the frame-related data stored at a designated intersection point of the sound dimension and the time dimension of the music framework.
19. A data structure for representing musical information comprising:
memory means for storing the musical information in a machine-readable format; and
a multi-dimensional framework having:
a time dmension for separating the musical information into a plurality of segments, each segment representing some portion of a measure; and
a sound dimension for separating the musical information into a plurality of channels, each channel representing an instrument,
such that a plurality of framework intersection points are defined by the combination of a time dimension value and a sound dimension value;
whereby the musical information for a given channel and segment is stored in the memory means at the framework intersection point corresponding to the time dimension value and the sound dimension value that correspond to the given channel and segment.
20. The data structure of claim 19 wherein the framework further comprises a performance dimension accessible at each framework intersection point for storing additional musical information representing one or more multiple renditions of a particular measure as played by a particular instrument.
21. A data structure for representing musical information comprising:
entry pool means for storing a plurality of entries that represent rhythmic, melodic, and interpretive aspects of the musical information, each entry comprising:
one unit of music data selectively representing a rest, a note or a chord;
a set of details associated with the entry; and
a pointer to a next sequential entry in a time sequence of the entries;
a plurality of frame records for designating the musical information associated with a given measure for a given instrument, each frame record having:
a first pointer to the entry pool for designating the first entry to be included in the measure; and
a last pointer to the entry pool for designating the last entry to be included in the measure.
22. The data structure of claim 21 wherein the entry pool means is a two dimensional double link list array having as a first dimension the time sequence of the entries and as a second dimension a relationship among entries occurring in a same time interval.
23. The data structure of claim 22 wherein each entry comprises:
a last pointer defining the previous entry in the time sequence;
a next pointer defining the subsequent entry in the time sequence;
an up pointer defining a first related entry occurring during the same time sequence;
a down pointer defining a second related entry occurring during the same time sequence;
a duration value defining the duration of the entry;
a position value defining the position of the entry;
a note value defining the pitch of the entry; and
an effect value defining a set of attributes that effect the appearance and functionality of the entry.
24. The data structure of claim 23 wherein the duration value defines a relative duration of the entry.
25. The data structure of claim 23 wherein the position value defines a relative horizontal position of the entry and musical staff.
26. The data structure of claim 23 wherein the note value defines a relative pitch of the entry.
27. A method for processing musical information comprising the steps of:
inputting the musical information into a programmable data processing means for electronically representing the musical information;
identifying a plurality of measures in the musical information;
assigning a time dimension value to each measure;
identifying at least one instrument in the musical information;
assigning a sound dimension value to each instrument;
storing the musical information for a given instrument and measure by storing the time dimension value and the sound dimension value assigned to the given instrument and measure and storing the musical information associated with the given instrument and measure.
28. The method for representing musical information of claim 27 further comprising the steps of:
identifying a plurality of entries that represent rhythmic, melodic, and interpretive aspects of the musical information, each entry comprising one unit of music data selectively representing a rest, a note or a chord and a set of details associated with the entry;
linking successive entries for a given instrument together;
designating the musical information to be associated with the given instrument and measure by;
assigning a first pointer to the successive entries for the given instrument to designate the first entry to be included in the measure; and
assigning a last pointer to the successive entries for the given instrument to designate the last entry to be included in the measure.DescriptionTECHNICAL FIELD
The present invention relates generally to the field of music processing devices and methods and apparatus for musical transcription and notation. More particularly, the present invention relates to a method and apparatus for representing within a common data structure in the memory means of a computer data processing system musical information that includes both the graphic or visual attributes of the musical information and the related acoustic or sound attributes of the musical information.
BACKGROUND ART
Like human language, musical information is comprised of both aural and written components. The written language of music or musical notation has existed for more than eight centuries, but until the advent of the printing press musicians and composers were required to perform the time consuming task of manual notation in order to memoralize their compositions. Even with the printing press, music notation processing has always been a post composition process usually performed by someone other than the composer or musician. With the introduction of computers, special programming languages have been developed to handle the entry and printing of musical notation. These languages generally use a textually-based user interface that requires the user to enter lengthy sets of computer codes in order to generate a single page of musical notation.
In recent years, music transcription systems have been developed for personal computers in an effort to aid the musician and composer in using musical notation. Some of these transcriptions systems have even attempted to take acoustic musical information recorded as electronic messages and turn this information into standard musical notation or sheet music. Unfortunately, the transcription process of these systems is generally a one-way path and there are no provisions for capturing interpretive musical information, or for recreating the musical information as sound based on the stored music data that are transcribed from the original musical information.
While the music processing devices presently available allow a user to print musical information more efficiently than traditional hand engraving of printing plates, they are far from an ideal music processing apparatus. An ideal music processing apparatus should be able to receive the musical information from its native environment (musical sound), represent the musical information in such a way that it may be automatically converted into its written language equivalent (musical notation), present the musical notation to the user so that it may be manipulated or edited, and then output the musical information, either as musical notation or as musical sound, exactly as it has been composed.
Present music processing apparatus cannot simulate an ideal music processing apparatus in large part because of the limitations imposed by the present methods and systems for internally representing musical information in these systems. The internal representation of musical information for a music processing apparatus is a problem that has challenged musicologists for more than twenty-five years with no single or satisfactory solution. Most music processing apparatus use a code-type representation that stores only the specific musical information needed to achieve the limited objectives of the system, i.e., information about pitch alone, about rhythm alone, or about pitch and rhythm without reference to dynamics or articulation. A few music processing apparatus have attempted to develop language-type representations that would enable the user to encode any element of musical information that is present in common musical notation. For example, the DARMS and MUSTRAN encoding languages were developed for mainframe computers. While flexible and more complete than the code-type representations, current music encoding languages are limited to textual-based entry of musical information and are not structured enough to provide a workable method and apparatus for representing musical information that could be used to create a device approaching the ideal music processing apparatus.
The Directory of Computer Assisted Research in Musicology, Center for Computer Assisted Research in the Humanities, 1987, pp. 1-22, identifies five broad classifications that encompass the current methods and systems for representing musical information: music/machine-readable code; music/logic; music/parametric; music/graphic; and music/acoustic. These classifications provide a background for thinking about the various stages of music data representation as the music data move from the input process to the various output processes of display, printing, analysis and sound in a music processing apparatus.
The first classification, music/machine-readable code, covers music processing apparatus that use a code-type representation, usually in the form of an unstructured character string, wherein each character represents, for example, the pitch or duration of the next role to be played in the sequence. The music/logical classification covers those representations that attempt to organize the musical information into logical records and, in this form, more closely represent the logical meaning or structure of the musical information itself. This is the general type of representation that is necessary for developing an ideal music processing apparatus. The next classification, music/parametric, includes all the information of the music/logical representation but presents this data as a list of objects (notes, rests, beams, etc.) whose attributes are determined by specific parameters. Most current music printing programs process their music data in this form. The last two classifications relate primarily to the output of musical information. The music/graphic classification covers representation forms that are closely related to the actual printing or display of the graphic musical information. Examples might include font-lists with X-Y coordinates or strings of redefined ASCII characters that translate directly into music graphics. The music/acoustic classification covers representation forms that are closely related to the actual generation of musical sound. The best known examples is the MIDI (Musical Instrument Digital Information) standard that is currently used by manufacturers of synthesizers and other electronic musical instruments. For a more detailed explanation of the MIDI format, reference is made to Boom, Music Through MIDI, 1987, Chapter 5, pp. 69-94, which is fully incorporated by reference herein.
Although the various music processing apparatus currently available have enabled music publishers to produce higher quality printed music or enabled hobbyists to enter and print simplistic musical notation, none of these systems has a method or device for representing musical information that enables a musician or composer to easily and accurately transcribe complete musical acoustic information into complete musical graphic information and to reverse this process. Accordingly, there is a continuing need for the development of new tools to assist the musician and composer in the transcription of musical information by providing a method and apparatus for representing musical information that will allow the musician or composer to enter, manipulate and retrieve both the graphic and acoustic attributes of musical information from a common data structure in the memory means of a computer data processing system, whereby changes to one type of information may be automatically reflected in changes to the other type of information.
SUMMARY OF THE INVENTION
In accordance with the present invention a method and apparatus for representing musical information are provided, utilizing a programmable data processing means having storage means for representing both acoustic and graphic musical information in a common data structure and input/output means operably connected to the data processing means for entering, editing and retrieving the musical information.
The method for electronically processing and storing musical information involves separating the musical information into a plurality of segments, each representing some portion of a measure, and assigning a sequential time dimension value to each segment and measure. The musical information is also separated into a plurality of channels, with each channel representing a sound source (e.g. an instrument) and having a sound dimension value assigned to it. The musical information for a given channel and segment is stored in a memory array by associating the musical information corresponding to a given channel and segment with the memory array node specified by the time dimension value and sound dimension value assigned to the given channel and measure in which the segment appears.
The method for representing musical information of the present invention may further comprise the steps of identifying a plurality of entries that represent certain rhythmic, melodic, and interpretive aspects of the musical information, with each entry comprising one unit of music data selectively representing a rest, a note or a chord. A set of details comprising further acoustic or graphic features of a rest, note or chord may be associated with the entry. Successive entries for a given instrument or sound source are linked together. The linked entries are then assigned to locations in the memory array by defining a first pointer to the successive entries for a given instrument or sound source to designate the first entry to be included in a measure and defining a last pointer to the successive entries for the given instrument or sound source to designate the last entry to be included in the measure.
A primary objective of the present invention is to provide a method and apparatus for representing musical information that will approach an ideal music processing apparatus.
Another primary objective of the present invention is to represented musical information in a music processing method and apparatus in a manner such that the musical information may be organized according to the rules of musical notation and interpretation as seen and heard by musicians.
An additional objective of the present invention is to provide a method and apparatus for storing music data specifying a composition in a form that readily permits either the output of graphic music notation or acoustic performance.
Another objective of the present invention is to provide a method and apparatus for storing musical at a in which the data structure used for storage permits easy access for editing the composition and facilitates automatic changing of graphic elements of music notation when acoustic information is changed and vice versa.
Another objective of the present invention is to provide a method and apparatus for storing music data in which essentially all forms of graphical symbols and text used in musical notation may be stored with a clear relationship to the corresponding acoustic information.
A further objective of the present invention is to provide a multi-dimensional data structure or memory array in which the music/logic relationship between notes, chords and rests and various interpretive markings is clearly represented.
A still further objective of the present invention is to provide a data structure for music data in which most data is expressed in a common format and pitch, duration and graphic location are all expressed as relative representations for easy transformation.
These and other objectives of the present invention will become apparent with reference to the drawings, the description of the preferred embodiment and the appended claims.
DESCRIPTION OF THE DRAWINGS
FIG. 1 is a diagram of a music processing apparatus in accordance with the present invention including an electronic instrument keyboard, a MIDI interface, and a programmable data processing means including an alphanumeric keyboard, a pointing device and a graphic display screen.
FIG. 2 is an overall block diagram of a Music Framework for representing musical information in a music processing apparatus in accordance with the present invention, including the Enigma Format of the present invention.
FIG. 3 is a block diagram of the Entry Pool of the present invention showing the relationships among Entries (note, rests, and chords) stored in the Music Framework.
FIG. 4 is a block diagram of the preferred embodiment of the Entry Pool showing the Entries in a two dimensional double link list arrangement for a simple six-entry Entry Pool.
FIG. 5 is a block diagram of the Relative Representation of measured music data in the present invention showing Tone Center Displacement for representing the pitch value of Entries and vertical graphic placement of a note on a staff as this information is stored together in the Music Framework.
FIG. 6 is a block diagram for the Frame Assignment method showing music data assigned to specific instruments and measures on the Music Framework.
FIG. 7 is a notational representation of the Music Framework showing selected measures of a sample composition associated with nodes in the Music Framework.
FIG. 8 is a block diagram of the Music Framework showing the relationship among various elements for the sample composition shown in FIG. 7.
FIGS. 9a and 9b are block diagrams showing the input and output of music data to the music processing apparatus of the present invention.
FIG. 10 is a depiction of a computer screen showing a Framework Dialog Box for interfacing with the Music Framework.
FIG. 11 is a block diagram showing the format of a transportable version of the music data represented in the Music Framework.
FIG. 12 is a block diagram of a Data Structure intensive Model showing the relationship among the various data structure groups used by the Music Framework.
DESCRIPTION OF THE PREFERRED EMBODIMENT
To understand the nature and scope of the present invention, it is first necessary to define the various types of information that comprise the musical information to be represented by the present invention. The present invention represents all of the musical information for a composition within a common data structure in the storage means of a computer data processing system. The logical relationships that are represented within this common data structure allow changes in one type of musical information to be reflected in all other types of musical information as needed. The various “types” of musical information that are represented within this one common data structure arise out of the natural representations of both graphic (visual) musical information and acoustic (sound) musical information and the logical relationships between the graphic and acoustic musical information (e.g., pitch expressed in terms of frequency and pitch expressed in terms of placement of a note on a staff), as well as the interrelationships among the natural representations of either graphic or acoustic musical information (e.g., duration as affected by a tie or pitch as affected by a slur). Briefly, the types of graphic and acoustic information that are represented in the common data structure used in the present invention are: melodic information, rhythmic information, interpretive information, and textual information.
As with languages, music is a way for humans to formulate, understand, manipulate and communicate information. Accordingly, the various representations of musical information are not dictated as much by the laws and limitations of nature as they are by the conventions and rules of musicians. The present invention attempts to understand and replicate the various interrelationships, conventions and rules that govern the way musicians see and hear music, rather than imposing a new set of rules and standards for how to represent music in a manner that is most convenient for a computer. In this way, the present invention should be viewed as extensible and able to adapt to new interpretations and methods of how humans represent music. The various types of information that are represented by the present invention are not meant to limit human imagination in creating new ways of representing or understanding musical information. Instead, the present invention encourages looking at musical information in a new light and allows for non-traditional methods of representing that information, for example twentieth century notation.
Though the present invention is not limited by any arbitrary division of the types of musical information that are represented in its common data structure, it is helpful to define the types of musical information that will be discussed. Melodic information refers primarily to both the pitch and absolute duration of the individual notes entered by the musician or composer. Pitch refers to the tonal properties of a sound that are determined by the frequencies of the sound waves that produce the individual note. In classical western musical notation, pitch is denoted with reference to a series of half-step intervals that are arranged together in octaves; each octave comprising 12 half-steps or notes. For purpose of defining melodic information as used in this invention, note duration is the length of time a particular note is played. Note duration is sometimes thought of as the relative time value of a given note, e.g., whole note, half note, quarter note, eighth note. For purposes of this invention, however, note duration in terms of melodic information refers only to the absolute time value of a given note, i.e., absolute note duration. It is necessary to distinguish between relative and absolute time value of a note, because relative time value can only be correctly resolved when the proper beat unit is known, i.e., a half note played at 160 beats per minute should be notated differently than a quarter note played at 80 beats per minute, even though both notes will have the same absolute time value.
Rhythmic information, on the other hand, refers to everything pertaining to the time and emphasis aspects of multiple notes as distinct from their melodic aspects. It includes the effects of beats, accents, measures, grouping of notes into beats, grouping of beats into measures and grouping of measures into phrases. For purposes of the present invention, four distinct components comprise the rhythmic information necessary to easily and accurately transcribe music into musical notation: (1) relative note duration–this is the length of time a note is played in terms of the time signature for the measure; i.e., half note, quarter note; (2) beat unit–the base unit of time used to measure the tempo of a piece of music; (3) measure–the organization of beat units into groups corresponding to the time signature of the composition or section of a composition; and (4) accent–the designation of particular emphasized beat units or notes within a measure. The function and importance of rhythmic information or the “beat” relates to the fact that the human ear seems to demand the perceptible presence of a unit of time that can be felt as grouping the individual notes together. In classical western notation, the beat unit and the relation between beat units and measures are designated by the tempo marking, e.g., 120 beats per minute, and the time signature, e.g., 3/4, where the top number indicates the number of beat units per measure (in this case 3) and the bottom number designates the type of note in which the beat units will be measured, i.e., the note value that will receive one beat unit (in this case a quarter note). Though sometimes referred to as the beat, for purposes of this invention, an accent will define which notes, beat unit(s), or sub-divisions of beat units in a measure or group of measures are to receive accentuation or emphasis.
Interpretive information refers to the characteristic sounds that are imparted to a piece of music when it is played with the expressions and feelings of a particular performer. Interpretive marks such as crescendos, staccatos, ritards, as well as information relating to tempo, other dynamics, and even settings for modulation wheels on electronic instruments. Interpretive information relates to the manner in which a particular performer will interpret a given piece of music.
Textual information refers to the language information that is associated with a given piece of musical information. This includes not only lyrics, but also title, related notes and other textual information, such as headers and footers, that are not included in the interpretive information.
While musical information may sometimes be referred to throughout this invention according to these four types, it should be understood that these categories are somewhat arbitrary and are used to convey an understanding of the overall concepts embodied in the invention. For example, some of the interpretive information might be just as easily referred to as textual information or rhythmic information, depending upon the usage and context of the musical information. The fundamental concept behind the present invention is that the musical information represented in a music processing apparatus be organized according to the rules of musical notation and interpretation as seen and heard by musicians.
THE MUSIC PROCESSING APPARATUS
Referring now to FIG. 1, the functional relationship among selected elements of the music processing apparatus of the present invention can be seen. The music processing apparatus 10 is comprised of a programmable data processing means 12 operably connected to associated input/output means, including an alphanumeric keyboard 16, a pointing device 18, a graphic display screen 20, an instrument means 22 with associated interface means 24, and a printer 26. The preferred programmable data processing means 12 of the present invention is a digital microcomputer having memory means (not shown) for representing the musical information in a common data structure as described hereinafter. In one preferred embodiment, the data processing means 12 is an Apple Macintosh SE computer available from Apple Computer, Inc. 20525 Mariani Avenue, Cupertino, Calif. 95014. The instrument means 22 shown in the preferred embodiment is an electronic keyboard, a DX-7 synthesizer available from Yamaha International Corporation, P.O. Box 6600, Buena Park, Calif. 90622. It will be seen, however, that any instrument equipped with an interface means 24 for encoding the musical information as MIDI or other machine-readable data would be capable of providing musical information to the music processing apparatus 10. It should also be noted that data processing means 12 may include the interface means 24 within the components included in the computer housing. In the preferred embodiment, the Apple Macintosh SE is equipped with an Apple MIDI interface unit and a Laser Writer printer, also available from Apple Computer, Inc. The functioning of the data processing means 12 is controlled by means of control information in the form of a software program that stores and retrieves data utilizing the common data structure described hereinafter. Those skilled in the art will recognize that various software functions utilizing the common data structure could be programmed in a variety of ways or could also be accomplished by equivalent hardware.
THE COMMON DATA STRUCTURE
The music processing apparatus 10 may be used by a musician or composer to enter, retrieve, edit, analyze and output musical information. In order to accomplish these tasks, the musical information must be stored in a machine-readable format so that it may be processed by the data processing means 12. For purposes of the present invention, musical information represented in this manner will be referred to as music data. The common data structure that the music processing apparatus 10 uses to represent music data in the memory means of the data processing means 12 is a new way of storing musical information in a machine-readable format such that the graphic aspects of the music data are related to the acoustic aspects of the music data in a music/logical type of representation. By accommodating both the graphic and acoustic aspects of the music data as related elements in a common data structure, changes to the graphic information can be automatically reflected in changes to the acoustic information, and vice versa. The common data structure allows the present invention to overcome the problems inherent in the prior art where the graphic and acoustic representations of music data are treated as separate and distinct pieces of information. The particular structure and organization of the common data structure also allow the present invention to overcome the problems inherent in the current systems that attempt to utilize a music/logical type of representation of musical information.
Unlike the prior art, the present invention does not define a specific coding scheme for representing each of the four types of musical information. Instead, a common data structure, a logical array called the Music Framework, is established in which elements of music data for any of the four types of musical information listed above may be represented. Moreover, these elements of music data are preferably represented in the logical array with the same basic format for each element, the Enigma Format. By using the same basic format or various types of music data and storing all of the music data in a common data structure, the storage and manipulation of the music data is simplified and the overall performance of the music processing apparatus is improved.
To further utilize the flexibility and other advantages of the common data structure (the Music Framework) and the basic format for storing music data (the Enigma Format) the present invention also provides for several novel methods and structures for representing the music data within the Music Framework. These begin with the way in which individual music data values are represented and extend to how groups of music data are placed in the Music Framework. First, although the Music Framework provides a method for relating instruments (or other sound sources) and measures (or portions of measures) in time, there is also a need to provide a time relationship among the notes, chord, and rests that are the heart of the music data for each measure. Because musicians and composers tend to create music data in a non-linear fashion, in the present invention the core of the music data (i.e., the notes, chords, and rests) is stored in a flexible, sequentially linked organization, the Entry Pool, that allows for easy insertion, deletion, or modification of the music data. Next, because certain types of music data are physical measured values (e.g., pitch, duration, position) that a musician may wish to systematically modify, as by transposition, rearrangement, or resizing, the present invention chooses to represent the individual value for any type of measured music data as a relative value expressed in terms of an offset or displacement from a user-selected or default absolute value, or expressed in terms of relative units multiplied by a base resolution value. This Relative Representation method allows for quicker manipulations and additional flexibility in representing the individual measured values of the music data, particularly for representation of pitch values by the Tone Center Displacement method. Finally, because composers often use the same music data in different measures and for different instruments, the present invention facilitates this reuse or tieing together of music data by using a Frame Assignment method to associate individual measures of music data for a particular instrument with nodes or intersection points in the Music Framework. This method also leads to certain storage and processing efficiencies in the music processing apparatus 10.
THE MUSIC FRAMEWORK
As shown in FIG. 2, all of the music data in the common data structure is organized within a Music Framework 30, a multi-dimensional array having Time 32 as its first dimension and Sound 34 as its second dimension. The Music Framework 30 is a kind of time-based organizational chart in which the method and apparatus of the invention places all the various types of musical information the music processing apparatus 10 must manage. Conceptually, the musical information is connected to the various Framework Intersections 36 created by the intersection of discrete values of the two dimensions, Time 32 and Sound 34, of the Music Framework 30. All of the music data represented in the memory means of the data processing means 12 is organized according to a music data hierarchy defined in association with the Music Framework 30. The hierarchical categories of music data represented in the Music Framework 30 include: Page-Related Data 40, Instrument-Related Data 42, and Measure-Related Data 44 represented in the Framework Margin 35, and Frame-Related Data 46 and Entry-Related Data 48 represented at the Framework Intersections 36.
Page-Related Data
Page-Related Data 40 includes all information about any specific page of a composition, including the dimensions of the page size, the page margins, and the system margins of any staff system that appears on the page. Also included in this category of music data are page, staff system, and individual staff reduction or enlargement parameters. Page-specific text blocks, arbitrary music, and headers and footers also are contained in the Page-Related Data 40 category.
Instrument-Related Data
Instrument-Related Data 42 is the musical information that is unique to a particular instrument or sound source (including a voice). This includes the name of the instrument, its transposition, and starting clef. In the Music Framework 30, when an instrument is assigned to a particular staff in a composition it is referenced by an instrument ID number. An unassigned instrument slot is simply a blank staff that is part of a staff system, but has not yet been assigned any frame records.
Measure-Related Data
Measure-Related Data 44 is the information related to each measure in the composition. For purposes of the present invention, the term “measure” applies to the particular measure number common to the music played by all of the instruments in the score during a specified time period. Included in this category is time dilation information, score expression information, repeat information, measure attribute information, measure number information, and key signature and time signature information.
Frame-Related Data
Frame-Related Data 46 is very much like Measure-Related Data 44. The difference between Frame-Related Data 46 and Measure-Related Data 44 is that the term measure is used to refer to specific measure for all instruments up and down the sound dimension of the Music Framework 30. By contrast, the term frame refers to just one measure or one portion of a measure for a particular instrument identified with a single Framework Intersection 36. Many of the Measure-Related Data 44 types of information, such as key and time signatures, are more easily thought of as frame-specific, because they may be varied among different instruments of the same measure. In the Music Framework 30, such data may be represented in either or both categories. For example, data relating to the basic key signature of a measure may be stored in the Measure-Related Data 44, and data for a special key signature for a particular instrument for that measure could also be stored in the Frame-Related Data 46. The Frame-Related Data 46 category includes data that are exclusively related to one frame, as opposed to a group of frames on different instruments. It includes MIDI expression information captured from a live performance, measure specific text blocks, and measure specific arbitrary music.
Entry-Related Data
Entry-Related Data 48 refers to the most basic elements in a composition. An entry may be either a note, a rest, or a chord. The music data that comprises the Entry-Related Data 48 represents the vast majority of the musical information to be processed by the music processing apparatus 10. Stored in the Entry-Related Data 48 is the rhythmic and melodic information for the entry, including the duration of the entry and the number of notes contained in that entry. Also stored here are attributes that are peripheral pieces of information attached to the entry. These may be lyrics, chords, staff expressions, or grace role definitions. A subset of Entry-Related Data 48 contains the displacement and accidental amount of the note, as well as tie information and key velocity information. A more complete definition of all the types of attributes found in the Entry-Related Data 48 category is provided in the explanation of the Entry Pool below.
THE ENIGMA FORMAT
FIG. 2 also shows the Enigma Format 50 that is the basic field format by which virtually all of the music data is stored in the memory means of the data processing means 12. The Enigma Format 50 is used to represent music data of all five of the hierarchical levels of music data and allows the data processing means 12 to break the musical information down into discrete segments of music data. Each segment of music data can be thought of as a data element that represents some definable portion of the musical information that has certain characteristics and attributes, as well as defined relationships with other data elements. The advantages of the Enigma Format 50 is that it does not limit what those data elements must be, nor does it limit how two or more data elements will be related to one another. Instead, the Enigma Format 50 provides a common basis for defining what each data element will represent and how that data element will relate to other data elements. In this sense, the Enigma Format 50 is truly extensible because a musician is not limited to the current way of defining or interrelating segments of music data and new ways of defining or interrelating music data elements may be easily and quickly incorporated into the common data structure for the music processing apparatus 10.
For each data element that is defined in the Music Framework 30 of data processing means 10, the Enigma Format 50 for that data element is comprised of three fields: a comparator 52, a tag 54, and an associated data field 56. The tag 54 is a code that identifies the type of data element. Each unique code to the tag 54 defines a unique type of data element, what the data element represents, and how the data element will relate to other data elements. For example, a `MS` tag identifies the data element as a Measure Specification tag having a defined data field associated with it. The particular combination of a comparator 52, a unique tage 54, and a defined data field 56 will be referred to hereinafter as a tag structure. The tag structure for a `MS` tag is based on this data element role as a part of the Measure-Related Data 44 and might include, for example, data defining the height and width of the measure when it is displayed on the graphic display 20 or printed on the printer 26. The comparator 52 is a numeric value that identifies how this data element is related sequentially to other data elements having the same type of tag 54. For example, a `2` comparator for a `MS` tag identifies the particular data element as the Measure Specification data element for the second measure of a particular composition. The data field 56 is a fixed-length packet of data whose content depends on the type of tag structure. For example, the data field 56 for an note may contain bit-mapped information that defines how to position a note on the staff, or for an element of Instrument-Related Data 44 it may contain ASCII codes representing the name of an instrument for a staff.
In the preferred embodiment of the Enigma Format 50, the comparator 52 may occupy two bytes of four bytes. The comparator 52 may be either a sixteen bit number with a maximum value of 65,535, or may be divided a 32 bit number divided into two 16 bit numbers each with a maximum value of 65,535. The tag 54 is a two character ASCII code. The length of the data field 56 is fixed, its length depending upon the tag structure identified by the two character tag 54.
THE ENTRY POOL
Because the bulk of the music data represented by the music processing apparatus 10 is comprised of Entry-Related Data 48 that must be accessed quickly and efficiently, the present invention uses a unique method and structure by storing the rhythmic and melodic music data associated with entries (notes, rests or chords). As shown in FIG. 3, the Entry-Related Data 48 is represented on the Music Framework 30 as an Entry Pool 60 comprised of rows of Entry Streams 62, 64, 66 and 68, each row made up of a plurality of Entry Elements 70. The Entry Streams 62, 64, 66 and 68 represent a sequential, time-based linking of the Entry Elements 70 that correspond to the Instrument-Related Data 42 for that row. These Entry Streams 62, 64, 66 and 68 are independent of one another. The Entry Pool 60 makes no stipulations on the concurrency or sequentially of the Entry Elements 70 as they are physically represented in the memory means of the data processing means 12. The key to understanding the structure of the Entry Pool 60 is that each Entry Element 70 in the Entry Stream 62 is linked to the next sequential Entry Element 70, as well as to any Entry Elements that belong to the same time interval, e.g., the individual notes of a chord. By using the sequential linking concept of the Entry Pool 60, the music processing apparatus 10 is able to represent the Entry-Related Data 48 as a series of threads or streams of entries that do not need to be stored in an exact physically sequential arrangement in the memory means of the data processing means 12.
In the preferred embodiment the structural foundation of the Entry Pool 60 is a two dimensional double link list. The two dimensional double link list structure for the Entry Pool 60 solves the problem of requiring an absolute identifier to access the Entry-Related Data 48. Because double link list data structures do not rely on physical sequentially for data access, this type of structure is well suited to the non-linear quick access requirements for real-time entry and manipulation of music data. Once an Entry Element 70 is defined as beginning at a certain offset, it need not shift physically in the memory means due to array manipulations because the double link list array manipulations do not move data, but merely rearrange links. Because only the links are manipulated during edits, the double link list provides a flexible structure in an interactive environment that meets the requirements for real-time entry and manipulation of music data. While a two dimensional double link list is described as the preferred embodiment for the Entry Pool 60 of the present invention, it should be recognized that other types of data structures permitting expression of sequential relationships may also be used to achieve similar sequential linked access to the Entry-Related Data 48 and are intended to be within the scope of the present invention.
Referring now to FIG. 4, the preferred embodiment of a two dimensional double link list structure for a single Entry Stream 62 of the Entry Pool 60 will be described. The first dimension is a time dimension (horizontal in FIG. 4) and the second dimension is a sound packing dimension (vertical in FIG. 4). The time dimension delineates the time relationship among sequential Entry Elements 70 in the Entry Stream 62. The sound packing dimension delineates the sound relationship among Entry Elements 70 in the Entry Stream 62 that occur during or start at the beginning of the same time period. The time dimension allows the Entry Element 70 to be sequentially linked to the Entry Stream 62. The sound packing dimension allows two or more Entry Elements 70 to be grouped together to form, for example, a chord.
In the example shown in FIG. 4, the individual Entry Elements 70 each have an associated Entry Number 71 (represented in the shaded ovals as Entry Nos. 1-5 and 8), although the Entry Number 71 is not actually a part of the data stored for the Entry Element 70. The Entry Number 71 is used to physically locate the Entry Element 70 in terms of a displacement or offset from a base value that represents the beginning of the Entry Pool 60 in the memory means of the data processing means 12. As described in conjunction with the section entitled Data Structure Intensive Model, the Entry Number 71 is also used as a comparator value to identify other tag structures that contain additional Entry-Related Data 48 that must be associated with the particular Entry Element 70 identified by the Entry Number 71.
The shaded rectangle in the top portion of FIG. 4 shows the data fields that are defined for each Entry Element 70, in this case by Entry No. 2. In essence, the Entry Pool 60 is comprised of an array of Entry Elements 70 that are each 32 byte elements containing these data fields. In the preferred programming language `C`, the data fields of the array consist of the following declared structure:
______________________________________ # define ENTRY struct entry ENTRY ______________________________________ { ufourbyte Last; ufourbyte Next; ufourbyte Up; ufourbyte Down; twobyte Duration; twobyte Position; fourbyte Effect; fourbyte Notevalue }; ______________________________________
The first four data fields defined in an the array (Last 72, Next 73, Up 74, and Down 75) are four byte values used to implement the preferred embodiment of the two dimensional double link list structure of the Entry Pool 60. Last 72 and Next 73 define the previous and succeeding Entry Elements 70. Up 74 and Down 75 provide for the construction of multiple notes or chords in the same time segment. Each of these variables contains the value of an Entry Number 71. By pointing to the next linked Element 70 in this manner, the Entry Elements 70 do not need to be stored contiguously in the memory means.
The two byte data field Duration 76 holds the relative duration of the entry in terms of relative