enode/README.md

ENODE: ENODE is Not an Oracle Development Environment

A package and 'mode' for providing an interface for examining and
developing for relational databases.

Drawing on TOAD by Quest Software as inspiration, this is provides an emacs
based interface to examine and develop for a relational database. It's
hoped that the following databases will ultimately supported:
  mysql
  postgres
  oracle
In fact, the proof-of-concept will be developed for an oracle database,
and as I will have reasonably easy access to mysql and postgres databases,
They will follow suit. I indend to provide high-level interfaces to
databases, so it will be easy to develop a layer for interacting with
databases from other vendors.

Initially, the following functionality will be provided:
  Interface -- look and feel.
  Connect and disconnect, plus management of saved connection information.
  Database object listing and examination.
  Area for typing of ad hoc SQL statements.
  Presentation of SQL query output.
  - There'll be no facility to change data as presented from a query.
    However, update, delete and insert commands will be facilitated
    through the SQL area.

Interface -- look and feel.
  The emacs frame will be divided into three windows:
  - The 'object list' or 'navigator' window.
;;         + Here will be listed in tree format the various objects that a
;;           schema has. Hierarchy will be something like: Schema, objects
;;           (tables, views, packages, etc.), columns/indexes (for tables and
;;           views) or procedures/functions (for packages), parameters, etc.
;;         + The user will enter and leave this window by key strokes or
;;           mouse clicks.
;;         + An object is selected by clicking with the middle button or
;;           hitting return when point is somewhere on its name. Once
;;           selected, information will be presented in the information frame
;;           (see below).
;;         + As a tree structure will be used, opening and closing of nodes
;;           will result from selection of the node as described. Selecting an
;;           'open' node will 'close' it and vice versa.
;;         + A node can be closed or opened using the left and right arrow
;;           keys. This will not result in selecting the node, therefore
;;           preserving the information presented.
;;         + This window can present any of a number of buffers. However, each
;;           of these buffers must be of a perticular type (e.g. navigator),
;;           and will have a local key map specific to its use. Separation of
;;           the buffers will facilitate management -- one buffer for tables,
;;           another for views, etc.
;;         + The top node for each buffer will be the name of the schema owner.
;;         + The user can toggle between showing only one schema owner in the
;;           list or all of them
;;     - The information window.
;;         + This will present information on the item from the navigation
;;           window that has most recently been selected.
;;         + The user can navigate to this window by way of key stroke or
;;           mouse click.
;;         + The window will provide a subset of all the information on the
;;           object, and a menu to facilitate showing other information.
;;         + This window can present any of a number of buffers. There will be
;;           a separate buffer type for each type of information being
;;           displayed (i.e. table columns are presented differently from a
;;           table's indexes, which is also different from the list of
;;           triggers on the table, etc.)
;;     - The SQL interaction window.
;;         + This window will present one of two buffers -- the SQL input
;;           buffer and the SQL result buffer.
;;         + They can both be presented, but at the expense of the other
;;           windows.
;;         + In the SQL input buffer, entering commands will be as simple as
;;           entering SQL commands in any sql-mode buffer.
;;         + Indentation will eventually be based on my preferred indentation
;;           scheme, as I am the dictator, and not necessarily benevolent.
;;         + Execution of the command will involve typing a key stroke rather
;;           than the RET key as we will want to format the command nicely.
;;         + The output buffer will present data in one of two formats:
;;             > Select commands will present the selected data in grid format.
;;             > Other commands will generate output in simply sequential
;;               output format.
;;     - Possible arrangements can include:
;;         + Three windows in two rows. The navigator and information windows
;;           in the upper row, the latter being the wider. The SQL interaction
;;           window being in the lower. The upper row would be the higher.
;;         + Three windows in two columns. The navigator in the first column,
;;           and the information and SQL interaction windows in the second.
;;           The latter column will be the wider and the information window
;;           will be higher than the SQL interaction window.
;;         + Two windows in either columnar or tiered format. The user decides.
;;           The windows will be related by function: The navigator and
;;           information windows together with the latter getting more space;
;;           the SQL input and output windows together, each getting equal or
;;           similar space. The SQL window can be displayed in one of the
;;           first two configurations if a function is called from the
;;           information window that warrants it.
;;     - Help information.
;;         + Help can be brought up by typing the '?' key. This will present
;;           the list of key strokes that perform tasks in the window which
;;           has focus.
;;         + The help display will be presented in the SQL interaction window,
;;           which will be presented if it isn't already.
;;         + If the focus is already in a buffer in the SQL interaction window,
;;           the help screen will be presented in the largest visible other
;;           window.
;;         + Typing the '?' key in the help buffer will replace its contents
;;           with all the keystrokes possible based on the type of buffer
;;           supported and listing key strokes that work all over ENODE.
;;         + The user can return to the buffer of most recent focus using
;;           a single key stroke.
;;         + The user can dismiss the help screen and replace the windows to
;;           their previous configuration by typing the 'q' key while in the
;;           help buffer.
;;           
;;
;; Connect and disconnect.
;;   - Upon startup, ENODE will ask for connection information in almost
;;     precisely the manner in which sql-.+ asks for it -- using the
;;     minibuffer to get the username, password and database information.
;;   - ENODE will save each connection information in a history file,
;;     and will maintain a completion list or lists to facilitate quick
;;     connection. For connections to new databases, ENODE will ask for
;;     the type of database (mysql, oracle, etc). This will be stored with
;;     the connection information.
;;   - The actual commands that will be executed against the database will
;;     be based on the type of database being used. However, this will
;;     mainly be hidden from the user.
;;   - ENODE will facilitate concurrent connections.
;;   - A list of possible connections can be presented in the navigation
;;     screen. Open connections will be marked. Opening a closed connection
;;     involved 'selecting' it. Closing an open connection should not be
;;     that easy, and will involve a key stroke followed by an 'Are you
;;     sure?' question. Selecting an open connection which is not the
;;     current connection makes that connection current.
;;     Each connection can be represented in this list either by an alias
;;     given to it explicitly by the user or by a connection string in the
;;     format of something like <user>/<database>@<host>
;;   - Switching between connections will be at the drop of key stroke.
;;       + It will be wise to figure out from the start how new connections
;;         effect the buffers being displayed at the time.
;;       + See above regarding switching between connections using the
;;         navigator window.
;;   - Closing connections can be done by one of two means:
;;       + Close the current connection. Done with a key stroke and a
;;         response to an 'Are you sure?' question, the next connection in
;;         the list of open connections will be activated. If we are closing
;;         the final connection ENODE will report this but not close the
;;         application.
;;       + Place the point in the connection in the navigator and execute a
;;         key stroke.
;;
;; Database object listing and examination.
;;   - The most useful window here will be the navigator. It will list the
;;     objects of interest in a tree structure. There will be separate lists
;;     for tables, views, indexes and stored procedure/functions/packages.
;;     tables will drill down to triggers, columns, indexes and constraints.
;;     Columns will drill down to triggers, indexes and constraints.
;;     Views will drill down similarly. Packages will drill down to specs
;;     and bodies. Specs will drill down to types/procedures/functions/etc.
;;     Bodies will drill down to functions/procedures. Functions/procedures
;;     will drill down to parameter lists and return types (where
;;     appropriate).
;;   - The types of information displayed and the information itself will
;;     depend on the selected item, examples of which are:
;;       + Tables
;;           > Data
;;           > Columns
;;           > Constraints
;;           > Indexes
;;           > Triggers
;;       + Views
;;           > Data
;;           > Columns
;;           > Source
;;       + Constraints
;;           > Tables/Columns
;;       + Packages/procedures/functions
;;           > Dependancies
;;           > Source
;;       + Triggers
;;           > Tables
;;           > Source
;;     In the case of views and tables, if we want to see data, it is to be
;;     displayed in the SQL interaction window.
;;
;; Area for typing of ad hoc SQL statements.
;;   - This will display the SQL input buffer.
;;   - SQL commands can be typed as free text into the buffer.
;;   - Using key strokes, certain actions can then be run on the command in
;;     the buffer: execute, parse/compile, explain execution plan, etc.
;;   - Depending on a prefix argument to each of the key strokes commands,
;;     they will be executed on the contents of the buffer, the SQL command
;;     the point is currently in or on the commands that are in the region.
;;   - It will be possible to save the contents to a file.
;;   - It will be possible to clear the contents in one go.
;;   - It will be possible to insert the contents of a file, either after
;;     point or by first clearing the buffer.
;;   - Inserting the contents of the file into the buffer will not mean
;;     visiting the file. That functionality will come later.
;;
;; Presentation of SQL (query) output.
;;   - For commands other than select statements, the output presented will
;;     be as if the commands had been run on the command line.
;;   - Output from queries will be presented in a grid manner, the
;;     configuration of which will be decided after some initial testing.
;;
;; Internals
;;   - ENODE will maintain many lists which will be used extensively. These
;;     will all be association lists. All the elements of these lists will be
;;     string values, not symbols. Depending on the case sensitivity of the
;;     database system, these will be case sensitive or not. The following
;;     are some of these lists:
;;       + Databases. This list will be populated with the first database we
;;         connect to. The variable describing the current database will
;;         contain a string value from this list.
;;       + Schema owners. There will be a separate list of schema owners
;;         per database. As we connect to databases afresh, the first two
;;         elements of this list will be the user we connect as and the
;;         system/root schema. The variable describing the current schema
;;         owner we're connected as will contain an element from this list.
;;         If the user requests to see any information pertaining to a schema
;;         owner (s)he is not connected as, this list is populated fully.
;;         This list can be refreshed by typing the refresh key stroke while
;;         a schema owner has been selected in the navigation window.
;;         Refreshing the list also refreshes its presentation in the
;;         navigation window.
;;       + Tables. There will be a separate list for each owner. This list
;;         will be populated for the current schema owner as we connect
;;         for the first time. It will be populated for other schema owners
;;         as we request information on a table owned by that schema owner.
;;         This list can be refreshed by typing the refresh key stroke while
;;         a table is selected in the navigation window.
;;       + Views. There will be a separate list for each owner. This list
;;         will be populated for the current schema owner as we connect for the
;;         first time. It will be populated for other schema owners as we
;;         request information on a view owned by that schema owner. This list
;;         can be refreshed by typing the refresh key stroke while a view is
;;         selected in the navigation window.
;;       + Constraints.
;;       + Columns. A list per table or view.
;;       + Indexes. A list per table.
;;       + Packages. A list per schema owner.
;;       + Procedures. A list per schema owner for non packaged procedures, a
;;         list per package for packaged.
;;       + Functions. A list per schema owner for non packaged functions, a
;;         list per package for packaged.
;;         
;;   - Refreshing a list.
;;     The following will happen when a command to refresh a list is called.
;;       1.  An empty list will be created.
;;       2.  The command to populate this new list will be executed.
;;       3.  The contents of the new list will be compared with the existing
;;           list and newer elements will be added to it. Elements that are
;;           in the old list and are missing from the new will be removed from
;;           the old.
;;       4.  If the eode-refresh-recursively variable is non-nil, then
;;           any sublists will also be refreshed. in this manner.
;;       5.  Elements of a list that can have a sublist but that sublist
;;           is nil at the time of the refresh will not have that list
;;           populated. I.e. we don't refresh a list that hasn't been populated
;;           yet.
;;     The following will be applied during a list refresh:
;;       1.  The node in the navigation list will be 'closed' before the
;;           refresh begins.
;;       2.  The node's parent node will be 'closed'.
;;       3.  After the refresh, the parent's node will be opened again.
;;       4.  If the node that had been selected at the time of the call to
;;           refresh exists after the refresh, the point is sent to it and
;;           it is explicitly 'selected'. If it doesn't, the node's parent
;;           node is 'selected'.
;;
;;   - Interacting with the database.
;;       + The main engine will be the sql-<engine> functionality that is
;;         provided as standard with GNU/Emacs distributions.
;;       + All commands will be run in the background and will use the
;;         comint-redirect-send-command* functionality.
;;       + Lists will be read from temporary buffers.
;;       + Presented SQL output will probably have outputting formatting
;;         specified for the actual SQL interpreter being used and send to
;;         the SQL output buffer.
;;
;;   - Context.
;;     There will be variables that will maintain:
;;       + The current database and the most recent one.
;;       + The current schema owner and the most recent one.
;;       + The current table/view/package/etc. and the most recent one.
;;         There will be a separate pair of variables for each type of
;;         object ENODE supports.
;;       + The current selected item. I.e. There is a table named "FOO" and
;;         an index named "BAR". Both the enode-current-table,
;;         enode-current-index and enode-selected-item are nil. The
;;         user navigates to "FOO" in the navigation window and selects it.
;;         enode-current-table and enode-selected-item are set to "FOO".
;;         enode-current-index is still nil. The user then navigates to the
;;         "BAR" index and selects it. enode-selected-item and
;;         enode-current-index are set to "BAR", but enode-current-table
;;         remains set to "FOO".
;;       + The previous selected item.
;;       + The current buffer and the previous one.
;;       + The current window and the previous one.
;;
;; A typical session might be:
;;     1. A user calls the command M-x enode.
;;     2. The user is asked to enter a connection string, and is presented with
;;        a default which is the most recently used connection.
;;     3. The user can accept the default, use M-p and M-n to scroll through
;;        a list of saved connections, type in a connection (using completion
;;        to assist) or type in a new connection name or type in '+'.
;;     4. If the default is accepted that connection string is used to connect.
;;        If a connection from the list is entered, it's used. If a connection
;;        name that isn't on the list is entered, the user wants to create a
;;        new connection with that name. If '+' is entered, the user wants
;;        to create a new connection but doesn't know what to call it.
;;     5. If one of the last two, the user is asked for the type of database
;;        we want to connect to.
;;     6. Based on that, the sql-<vendor> function is called and the user is
;;        then prompted to enter the appropriate information.
;;     7. Once connected, the windows are initialised based on the user's
;;        customisation.
;;     8. The list of databases is initialised. The current database is set.
;;     9. The list of schema owners is initialised. If the user prefers to see
;;        all the schema owners in the navigation window at once, this list is
;;        populated with all the schema owners in the database. If not, this
;;        list is initialised to the requested schema owner and the
;;        system/root user. The current schema owner is set.
;;    10. The point is brought to the schema owner and the information for
;;        that user is presented in the information window. enode-selected-item
;;        is set.