enode/README.md

* ENODE: ENODE is Not an Oracle Development Environment

  A package and 'mode' for providing an interface for examining and
  developing for relational databases.

  Drawing on TOAD by Quest Software as inspiration, this is provides
  an emacs based interface to examine and develop for a relational
  database. It's hoped that the following databases will ultimately
  supported:
  - mysql
  - postgres
  - oracle

  In fact, the proof-of-concept will be developed for an oracle
  database, and as I will have reasonably easy access to mysql and
  postgres databases, They will follow suit. I indend to provide
  high-level interfaces to databases, so it will be easy to develop a
  layer for interacting with databases from other vendors.

  Initially, the following functionality will be provided:
  - Interface -- look and feel.
  - Connect and disconnect, plus management of saved connection
    information.
  - Database object listing and examination.
  - Area for typing of ad hoc SQL statements.
  - Presentation of SQL query output.
    + There'll be no facility to change data as presented from a query.
      However, update, delete and insert commands will be facilitated
      through the SQL area.

** Interface -- look and feel.
   The emacs frame will be divided into three windows:
   - The 'object list' or 'navigator' window.
     + Here will be listed in tree format the various objects that a
       schema has. Hierarchy will be something like: Schema, objects
       (tables, views, packages, etc.), columns/indexes (for tables
       and views) or procedures/functions (for packages), parameters,
       etc.
     + The user will enter and leave this window by key strokes or
       mouse clicks.
     + An object is selected by clicking with the middle button or
       hitting return when point is somewhere on its name. Once
       selected, information will be presented in the information
       frame (see below).
     + As a tree structure will be used, opening and closing of nodes
       will result from selection of the node as described. Selecting
       an 'open' node will 'close' it and vice versa.
     + A node can be closed or opened using the left and right arrow
       keys. This will not result in selecting the node, therefore
       preserving the information presented.
     + This window can present any of a number of buffers. However,
       each of these buffers must be of a perticular type
       (e.g. navigator), and will have a local key map specific to its
       use. Separation of the buffers will facilitate management --
       one buffer for tables, another for views, etc.
     + The top node for each buffer will be the name of the schema
       owner.
     + The user can toggle between showing only one schema owner in
       the list or all of them
   - The information window.
     + This will present information on the item from the navigation
       window that has most recently been selected.
     + The user can navigate to this window by way of key stroke or
       mouse click.
     + The window will provide a subset of all the information on the
       object, and a menu to facilitate showing other information.
     + This window can present any of a number of buffers. There will
       be a separate buffer type for each type of information being
       displayed (i.e. table columns are presented differently from a
       table's indexes, which is also different from the list of
       triggers on the table, etc.)
   - The SQL interaction window.
     + This window will present one of two buffers -- the SQL input
       buffer and the SQL result buffer.
     + They can both be presented, but at the expense of the other
       windows.
     + In the SQL input buffer, entering commands will be as simple as
       entering SQL commands in any sql-mode buffer.
     + Indentation will eventually be based on my preferred
       indentation scheme, as I am the dictator, and not necessarily
       benevolent.
     + Execution of the command will involve typing a key stroke
       rather than the RET key as we will want to format the command
       nicely.
     + The output buffer will present data in one of two formats:
       * Select commands will present the selected data in grid
         format.
       * Other commands will generate output in simply sequential
         output format.
   - Possible arrangements can include:
     + Three windows in two rows. The navigator and information
       windows in the upper row, the latter being the wider. The SQL
       interaction window being in the lower. The upper row would be
       the higher.
     + Three windows in two columns. The navigator in the first
       column, and the information and SQL interaction windows in the
       second.  The latter column will be the wider and the
       information window will be higher than the SQL interaction
       window.
     + Two windows in either columnar or tiered format. The user
       decides.  The windows will be related by function: The
       navigator and information windows together with the latter
       getting more space; the SQL input and output windows together,
       each getting equal or similar space. The SQL window can be
       displayed in one of the first two configurations if a function
       is called from the information window that warrants it.
   - Help information.
     + Help can be brought up by typing the '?' key. This will present
       the list of key strokes that perform tasks in the window which
       has focus.
     + The help display will be presented in the SQL interaction
       window, which will be presented if it isn't already.
     + If the focus is already in a buffer in the SQL interaction
       window, the help screen will be presented in the largest
       visible other window.
     + Typing the '?' key in the help buffer will replace its contents
       with all the keystrokes possible based on the type of buffer
       supported and listing key strokes that work all over ENODE.
     + The user can return to the buffer of most recent focus using a
       single key stroke.
     + The user can dismiss the help screen and replace the windows to
       their previous configuration by typing the 'q' key while in the
       help buffer.
          
** Connect and disconnect.
   - Upon startup, ENODE will ask for connection information in almost
     precisely the manner in which sql-.+ asks for it -- using the
     minibuffer to get the username, password and database
     information.
   - ENODE will save each connection information in a history file,
     and will maintain a completion list or lists to facilitate quick
     connection. For connections to new databases, ENODE will ask for
     the type of database (mysql, oracle, etc). This will be stored
     with the connection information.
   - The actual commands that will be executed against the database
     will be based on the type of database being used. However, this
     will mainly be hidden from the user.
   - ENODE will facilitate concurrent connections.
   - A list of possible connections can be presented in the navigation
     screen. Open connections will be marked. Opening a closed
     connection involved 'selecting' it. Closing an open connection
     should not be that easy, and will involve a key stroke followed
     by an 'Are you sure?' question. Selecting an open connection
     which is not the current connection makes that connection
     current.  Each connection can be represented in this list either
     by an alias given to it explicitly by the user or by a connection
     string in the format of something like <user>/<database>@<host>
   - Switching between connections will be at the drop of key stroke.
     + It will be wise to figure out from the start how new
       connections effect the buffers being displayed at the time.
     + See above regarding switching between connections using the
       navigator window.
   - Closing connections can be done by one of two means:
     + Close the current connection. Done with a key stroke and a
       response to an 'Are you sure?' question, the next connection in
       the list of open connections will be activated. If we are
       closing the final connection ENODE will report this but not
       close the application.
     + Place the point in the connection in the navigator and execute
       a key stroke.

** Database object listing and examination.
   - The most useful window here will be the navigator. It will list
     the objects of interest in a tree structure. There will be
     separate lists for tables, views, indexes and stored
     procedure/functions/packages.  tables will drill down to
     triggers, columns, indexes and constraints.  Columns will drill
     down to triggers, indexes and constraints.  Views will drill down
     similarly. Packages will drill down to specs and bodies. Specs
     will drill down to types/procedures/functions/etc.  Bodies will
     drill down to functions/procedures. Functions/procedures will
     drill down to parameter lists and return types (where
     appropriate).
   - The types of information displayed and the information itself
     will depend on the selected item, examples of which are:
     + Tables
       * Data
       * Columns
       * Constraints
       * Indexes
       * Triggers
     + Views
       * Data
       * Columns
       * Source
     + Constraints
       * Tables/Columns
     + Packages/procedures/functions
       * Dependancies
       * Source
     + Triggers
       * Tables
       * Source
     In the case of views and tables, if we want to see data, it is to
     be displayed in the SQL interaction window.

** Area for typing of ad hoc SQL statements.
   - This will display the SQL input buffer.
   - SQL commands can be typed as free text into the buffer.
   - Using key strokes, certain actions can then be run on the command
     in the buffer: execute, parse/compile, explain execution plan,
     etc.
   - Depending on a prefix argument to each of the key strokes
     commands, they will be executed on the contents of the buffer,
     the SQL command the point is currently in or on the commands that
     are in the region.
   - It will be possible to save the contents to a file.
   - It will be possible to clear the contents in one go.
   - It will be possible to insert the contents of a file, either
     after point or by first clearing the buffer.
   - Inserting the contents of the file into the buffer will not mean
     visiting the file. That functionality will come later.

** Presentation of SQL (query) output.
   - For commands other than select statements, the output presented
     will be as if the commands had been run on the command line.
   - Output from queries will be presented in a grid manner, the
     configuration of which will be decided after some initial
     testing.

** Internals
   - ENODE will maintain many lists which will be used
     extensively. These will all be association lists. All the
     elements of these lists will be string values, not
     symbols. Depending on the case sensitivity of the database
     system, these will be case sensitive or not. The following are
     some of these lists:
     + Databases. This list will be populated with the first database
       we connect to. The variable describing the current database
       will contain a string value from this list.
     + Schema owners. There will be a separate list of schema owners
       per database. As we connect to databases afresh, the first two
       elements of this list will be the user we connect as and the
       system/root schema. The variable describing the current schema
       owner we're connected as will contain an element from this
       list.  If the user requests to see any information pertaining
       to a schema owner (s)he is not connected as, this list is
       populated fully.  This list can be refreshed by typing the
       refresh key stroke while a schema owner has been selected in
       the navigation window.  Refreshing the list also refreshes its
       presentation in the navigation window.
     + Tables. There will be a separate list for each owner. This list
       will be populated for the current schema owner as we connect
       for the first time. It will be populated for other schema
       owners as we request information on a table owned by that
       schema owner.  This list can be refreshed by typing the refresh
       key stroke while a table is selected in the navigation window.
     + Views. There will be a separate list for each owner. This list
       will be populated for the current schema owner as we connect
       for the first time. It will be populated for other schema
       owners as we request information on a view owned by that schema
       owner. This list can be refreshed by typing the refresh key
       stroke while a view is selected in the navigation window.
     + Constraints.
     + Columns. A list per table or view.
     + Indexes. A list per table.
     + Packages. A list per schema owner.
     + Procedures. A list per schema owner for non packaged
       procedures, a list per package for packaged.
     + Functions. A list per schema owner for non packaged functions,
       a list per package for packaged.
        
   - Refreshing a list.

     The following will happen when a command to refresh a list is called.
     1. An empty list will be created.
     2. The command to populate this new list will be executed.
     3. The contents of the new list will be compared with the
        existing list and newer elements will be added to it. Elements
        that are in the old list and are missing from the new will be
        removed from the old.
     4. If the eode-refresh-recursively variable is non-nil, then any
        sublists will also be refreshed. in this manner.
     5. Elements of a list that can have a sublist but that sublist is
        nil at the time of the refresh will not have that list
        populated. I.e. we don't refresh a list that hasn't been
        populated yet.

     The following will be applied during a list refresh:

     1. The node in the navigation list will be 'closed' before the
        refresh begins.
     2. The node's parent node will be 'closed'.
     3. After the refresh, the parent's node will be opened again.
     4. If the node that had been selected at the time of the call to
        refresh exists after the refresh, the point is sent to it and
        it is explicitly 'selected'. If it doesn't, the node's parent
        node is 'selected'.

   - Interacting with the database.
     + The main engine will be the sql-<engine> functionality that is
       provided as standard with GNU/Emacs distributions.
     + All commands will be run in the background and will use the
       comint-redirect-send-command* functionality.
     + Lists will be read from temporary buffers.
     + Presented SQL output will probably have outputting formatting
       specified for the actual SQL interpreter being used and send to
       the SQL output buffer.

   - Context.

     There will be variables that will maintain:
     + The current database and the most recent one.
     + The current schema owner and the most recent one.
     + The current table/view/package/etc. and the most recent one.
       There will be a separate pair of variables for each type of
       object ENODE supports.
     + The current selected item. I.e. There is a table named "FOO"
       and an index named "BAR". Both the enode-current-table,
       enode-current-index and enode-selected-item are nil. The user
       navigates to "FOO" in the navigation window and selects it.
       enode-current-table and enode-selected-item are set to "FOO".
       enode-current-index is still nil. The user then navigates to
       the "BAR" index and selects it. enode-selected-item and
       enode-current-index are set to "BAR", but enode-current-table
       remains set to "FOO".
     + The previous selected item.
     + The current buffer and the previous one.
     + The current window and the previous one.

** A typical session might be:
   1. A user calls the command M-x enode.
   2. The user is asked to enter a connection string, and is presented
      with a default which is the most recently used connection.
   3. The user can accept the default, use M-p and M-n to scroll
      through a list of saved connections, type in a connection (using
      completion to assist) or type in a new connection name or type
      in '+'.
   4. If the default is accepted that connection string is used to
      connect.  If a connection from the list is entered, it's
      used. If a connection name that isn't on the list is entered,
      the user wants to create a new connection with that name. If '+'
      is entered, the user wants to create a new connection but
      doesn't know what to call it.
   5. If one of the last two, the user is asked for the type of
      database we want to connect to.
   6. Based on that, the sql-<vendor> function is called and the user
      is then prompted to enter the appropriate information.
   7. Once connected, the windows are initialised based on the user's
      customisation.
   8. The list of databases is initialised. The current database is
      set.
   9. The list of schema owners is initialised. If the user prefers to
      see all the schema owners in the navigation window at once, this
      list is populated with all the schema owners in the database. If
      not, this list is initialised to the requested schema owner and
      the system/root user. The current schema owner is set.
   10. The point is brought to the schema owner and the information
       for that user is presented in the information
       window. enode-selected-item is set.