Use Cases
It is helpful to describe some specific uses that the editor would be put to and to see what sort of functionality would be needed for each.
These might include a mail reader, a program editor, a web browser, a word processor with built in spreadsheet, a command-line window, a file browser, etc.
The common thread here is that data is stored somewhere, and displayed and manipulated through this universal editor.
Mail Reader
A mail reader uses both highly structured and (almost) unstructured data. It presents a collection of mail items each of which have well defined (though not completely defined) headers and body. The body is often largely unstructured, though MIME does give some structure to it sometimes.
A mail reader may need to present multiple items from multiple files, and needs to present a simple summary of these items with varying sorting or selection. It might also need to present a list of mail boxes, possibly in a tree formation.
These concepts are all highly structured so the mail reader will maintain a lot of control. Some editing will be possible, but it would be minimal, limited to setting or clearing specific attributes, moving items between mail boxes, and deleting items.
The mail reader also needs to present individual items which would have content that is normally not editable.
Finally the mail reader needs to provide for composing new message which allows for lots of editted to be done, including formatting, spell-checking and any other operation that could apply to 'plain text'.
To best provide for these needs, the mail reader should parse the mail messages into a data structure that makes the headers easily available, and particularly makes available those attributes that are used for sorting and selection. Quite possibly there would be a predefined set of selections (virtual mail boxes) and each message would be tags with a list of virtual mailboxes that it is a member of. New messages would be automatically tags. When a new virtual mail box were defined, all current messages would be checked and tagged accordingly.
It might be best to have a + or - tag so that lazy tagging can happen. i.e. when viewing the message list for a new virtual mailbox, only tag until there are enough to fill the screen, then tag the rest lazily. This would make message counts are to keep accurate.
Each different virtual mail box would be represented by a window that knew how to display a message list. Each would have it's own tag to check for and would only display message with that tag. Each window could also identify which fields to display and how to display them.
The mail items themselves would be displayed in a separate window (or pane). There would be different configurations for these windows depending on whether raw-text were preferred or formated text/images. Each display window would potentially be a scolling list of pains with different multipart/mixed sections in different panes, and each pane might be able to switch between various multipart/alternate options.
If a section were of a type that was not supported natively, it could optionally be supported with an embedded window for an external client. Alternately, there could be a control which started and independant extenal client.
Each pane should be selectable between raw-text, internal-formatting, or any of various external handlers. This suggests that they should be full-fledged editor windows which are contains within another editor window. As a section of the mail message can itself be a mail message with substructure we need to be prepared to have panes within panes within panes.
When composing messages, there need to be windows for editing some of the headers like From: and To: etc. These should be little light-weight windows (panes) that have their own behaviour and auto-syntax-help.
There also needs to be a main pane for editting the message This might need to be multiple panes if attachments are added in separate panes. You might even want to edit the non-primary message pane.
We want to allow creation of rich-text in messages (despite what our personal opinion of rich-text email might be). Thus any editting mode needs to able to be embedded in a mail message. Editting the body of a mail message should be no different from editing a normal text file.
The headers should be kept clearly separate. They need a lot of flexability for editting - anything should be possible - but they should be kept clearly distinct from the body.
Program Editor
The most interesting thing that a program editor can do for the programmer is to help with syntax and semantic.
Syntax help can involve highlighting errors and providing suggestions for what to type next, such as the rest of the variable name or the closing backet.
Semantic help (or is it really high-level syntactic help) could involve locating the definiation of a name from it's use and vice-versa.
There is a tension in deciding how much syntax to parse and store. On the one hand it would be nice for the editor to have full knowledge of syntax so that very useful help can be given. On the other hand programmers are used to editting the text, and anything that gets in the way of manipulating plain text would be a problem.
Storing a fully-parsed program would probably require the programmer to perform tree-based transformations at times when they really want to perform text based transformations which have tree base implications. Storing the program as raw text means that there will often be syntactic errors or inconsistancies which the editor will need to cope gracefully with.
It would be nice to find a middle ground where the program is parsed but the text is still all allowed to be editted. When editing causes syntax errors, the smallest enclosing program block is reparsed on each edit so local syntax issues will be highlighted. If the parse fails to successfully parse the whole block of text, the failed parse remains local until the cursor moves outside the parsed block, at which point a larger section of text is re-parsed.
It might be good to store the parse 'beside' the text rather than breaking the text into parsed units. Possible for smaller units like expression, this should always be done while for larger unit (functions and compound statements) the text could be broken into the parsed chunks for easy manipulation.
Having a strong parse would make it easier to store and maintain symbol tables that can be used to assist with naming.
I think the main point here is that it would be good to be able to hold a parse-tree that parallels the text without directly interfering with it. Nodes in the parse tree need to anchor to points in the text and move with them. Changes in the text need to be efficiently mirrored in the parse tree.
Web Browser
A Web Browser is really hard. HTML isn't too bad, even with style sheets. But javascript makes it an order of magnitude more difficult and Java and Flash just start to look silly.
I think that for a web browser embedded in an editor, Java and Flash are simply out of the question. To view a web page containing there, you just pass the address to some other program. Javascript might be managable if it could be auto-translated into suitable python. But the focus should be on HTML and other more static document types.
Why embed a browser in the editor? One reason that jumps out at me is that a browser needs a text editor, for those