various explanatory text

version number in __version__

AURA copyrights

remove code to support importing runtime instead of incorporating it into the generated parser

add MANIFEST.in

instructions for re-generating the parser that exyapps uses internally



git-svn-id: http://svn.stsci.edu/svn/ssb/etal/exyapps/trunk@431 d34015c8-bcbb-4646-8ac8-8ba5febf221d

9 files changed, 104 insertions, 53 deletions

diff --git a/MANIFEST.in b/MANIFEST.in
new file mode 100644
index 0000000..05c250d
--- /dev/null
+++ b/MANIFEST.in
@@ -0,0 +1,9 @@
+recursive-include examples *.g
+recursive-include exyapps *
+recursive-include scripts *
+recursive-include test *
+recursive-include doc *
+include LICENSE Makefile README setup.py  yapps_grammar.g
+include examples/notes
+
+# drwxrwsr-x 3 sienkiew ssb 4096 Aug 31 15:05 doc
diff --git a/Makefile b/Makefile
index 3c1f2a6..87821e4 100644
--- a/Makefile
+++ b/Makefile
@@ -1,2 +1,8 @@
 exyapps/grammar.py: yapps_grammar.g
 	exyapps yapps_grammar.g exyapps/grammar.py
+
+clean:
+	rm -rf dist build
+	rm -f MANIFEST
+	rm -f examples/calc.py  examples/expr.py  examples/lisp.py  examples/xml.py
+
diff --git a/README b/README
index a92e1ff..1cf78e3 100644
--- a/README
+++ b/README
@@ -3,11 +3,20 @@ This is exyapps, a LL(1) parser generator.
 It is derived from yapps ( http://theory.stanford.edu/~amitp/yapps/ )
 by Amit J. Patel <amitp@cs.stanford.edu>.  He is no longer maintaining
 it, and there seem to be several forks out there, all with varying
-version numbers.  This copy was derived from some patches included
-with Debian by Matthias Urlichs <smurf@debian.org>
+version numbers.  Matthias Urlichs <smurf@debian.org> made some patches
+for Debian; this copy was derived from the Debian distribution by Mark
+Sienkiewicz at the Space Telescope Science Institute.  (For email,
+use the first 8 letters of my last name and @stsci.edu)
+
+Some of the modifications that changed yapps to exyapps introduced the
+possibility of a fundmamental incompatibility with existing yapps2-based
+parsers.  This, coupled with the non-linear version numbers of the
+various forks, prompted me to rename yapps to exyapps.
+
+(STScI is a subsidiary of the Association of Universities for Research
+in Astronomy, which is why the copyright to the new code in exyapps
+belongs to AURA.)
 
-Since I want to make some of my own specific changes, I'm actually changing 
-the name so this package can have a distinctive identity.  
 
 
 Installing / using exyapps
@@ -19,15 +28,28 @@ Installing / using exyapps
 
         exyapps my_grammar.exy
 
-For now, the package is compatible with yapps; Someday, I want to
-make parsers run without the exyapps package installed.
+Modifying exyapps
+--
+
+The exyapps parser is written in exyapps.  If you want to modify
+yapps_grammar.g, use this procedure:
+
+	- install the current version of exyapps
+
+	- rm exyapps/grammar.py
+
+	- make
+
+This will re-generate grammar.py, which will be your new parser.  You can
+install it somewhere else to try it.
+
 
 VIM
 --
 
 Put this in .vimrc
 
-autocmd BufRead,BufNewFile *.exy                set filetype=python
+	autocmd BufRead,BufNewFile *.exy                set filetype=python
 
 What is here?
 --
@@ -35,21 +57,22 @@ What is here?
 Makefile
 yapps_grammar.g
         yapps_grammar.g is the source code for exyapps/grammar.py
-        type "make" to re-generate it, then do an svn commit
 
 doc
-        looks like latex source for the documentation
+        latex source for the documentation
 
 examples
-        duh
 
 exyapps
-        the exyapps package that gets installed
+        the exyapps package that gets installed - as of exyapps 3.0,
+        this is only need to compile the parser; you do not need to
+        install exyapps to run a generated parser.
 
 scripts
         "exyapps" command that compiles a parser into python code.
 
 setup.py
+	regular setup.py using distutils
 
 test
         not actual tests, but apparently some interesting input to
@@ -59,6 +82,9 @@ test
 New Features
 --
 
+- The generated parser no longer needs to have exyapps installed at
+run time.  The entire runtime is incorporated into the parser.  
+
 - You can pass a data object to the parser for it to use as
 parser-global data.  I know the OO way is to subclass the parser
 object and hope you don't accidentally override/smash anything
diff --git a/exyapps/__init__.py b/exyapps/__init__.py
index 1bb8bf6..eee103e 100644
--- a/exyapps/__init__.py
+++ b/exyapps/__init__.py
@@ -1 +1 @@
-# empty
+__version__ = '3.0dev'
diff --git a/exyapps/main.py b/exyapps/main.py
index 322a3eb..61f3478 100755
--- a/exyapps/main.py
+++ b/exyapps/main.py
@@ -1,15 +1,13 @@
 #!/usr/bin/python
 
 #
-# Yapps 2 - yet another python parser system
+# exyapps - yet another python parser system
 # Copyright 1999-2003 by Amit J. Patel <amitp@cs.stanford.edu>
+# Copyright 2011 by Association of Universities for Research in Astronomy
 #
-# This version of Yapps 2 can be distributed under the
-# terms of the MIT open source license, either found in the LICENSE file
-# included with the Yapps distribution
-# <http://theory.stanford.edu/~amitp/yapps/> or at
+# This software can be distributed under the terms of the MIT
+# open source license, either found in the LICENSE file or at
 # <http://www.opensource.org/licenses/mit-license.php>
-#
 
 import sys, re
 
diff --git a/exyapps/parsetree.py b/exyapps/parsetree.py
index 5e38665..964a74e 100644
--- a/exyapps/parsetree.py
+++ b/exyapps/parsetree.py
@@ -1,12 +1,16 @@
-# parsetree.py, part of Yapps 2 - yet another python parser system
+# parsetree.py, part of exyapps
 # Copyright 1999-2003 by Amit J. Patel <amitp@cs.stanford.edu>
+# Copyright 2011 by Association of Universities for Research in Astronomy
 #
-# This version of the Yapps 2 Runtime can be distributed under the
-# terms of the MIT open source license, either found in the LICENSE file
-# included with the Yapps distribution
-# <http://theory.stanford.edu/~amitp/yapps/> or at
+# This software can be distributed under the terms of the MIT
+# open source license, either found in the LICENSE file or at
 # <http://www.opensource.org/licenses/mit-license.php>
 #
+# -----
+#
+# Except for changes necessary to incorporate the runtime library directly
+# into the generated parser, this file is not substantially changed from
+# YAPPS 2.
 
 """Classes used to represent parse trees and generate output.
 
@@ -15,14 +19,16 @@ of Python output from a grammar parse tree.  It also defines nodes
 used to represent the parse tree; they are derived from class Node.
 
 The main logic of Yapps is in this module.
+
+This module is used only to compile a parser.
 """
 
 import sys, re
 
 import os.path
 
-# load the contents of the runtime file into memory; it will be
-# incorporated into the parser later.
+# Load the contents of the runtime file into memory.  We always need
+# this.  It will be incorporated into the generated parser code later.
 runtime_py_filename = os.path.dirname(__file__)+"/runtime.py"
 f = open( runtime_py_filename ,"r")
 runtime_contents = f.read()
@@ -42,7 +48,6 @@ class Generator:
         self.options = options
         self.preparser = ''
         self.postparser = None
-        self.inline_runtime = 1
         
         self.tokens = {} # Map from tokens to regexps
         self.ignore = {} # List of token names to ignore in parsing, map to statements
@@ -286,12 +291,9 @@ class Generator:
         self.write("import sys, re\n")
 
         # 
-        if self.inline_runtime :
-            self.write("###### included from %s\n"%runtime_py_filename)
-            self.write(runtime_contents)
-            self.write("###### end of runtime.py\n")
-        else :
-            self.write("from exyapps.runtime import SyntaxError, NoMoreTokens, Token, Scanner, Parser, Context, print_error, wrap_error_reporter\n")
+        self.write("###### included from %s\n"%runtime_py_filename)
+        self.write(runtime_contents)
+        self.write("###### end of runtime.py\n")
 
         self.write("\n")
 
diff --git a/exyapps/runtime.py b/exyapps/runtime.py
index bcbf253..6c984b8 100644
--- a/exyapps/runtime.py
+++ b/exyapps/runtime.py
@@ -1,13 +1,23 @@
-# Yapps 2 Runtime, part of Yapps 2 - yet another python parser system
+# exyapps - yet another python parser system
 # Copyright 1999-2003 by Amit J. Patel <amitp@cs.stanford.edu>
 # Enhancements copyright 2003-2004 by Matthias Urlichs <smurf@debian.org>
+# Copyright 2011 by Association of Universities for Research in Astronomy
 #
-# This version of the Yapps 2 Runtime can be distributed under the
-# terms of the MIT open source license, either found in the LICENSE file
-# included with the Yapps distribution
-# <http://theory.stanford.edu/~amitp/yapps/> or at
+# This software can be distributed under the terms of the MIT
+# open source license, either found in the LICENSE file or at
 # <http://www.opensource.org/licenses/mit-license.php>
 #
+# -----
+#
+# Changes from the debian version:
+# - indented with spaces, not tabs
+# - when you instantiate the parser object, you can pass in an object
+#   data=XXX that will be available to the parser as self.data; this
+#   is basically a hook to provide parser-global data.
+#
+# Note that this file is incorporated directly into the generated parser.
+# Take care when defining new file-global names because the generated
+# parser has its own globals.
 
 """Run time libraries needed to run parsers generated by Yapps.
 
diff --git a/scripts/exyapps b/scripts/exyapps
index a0d610d..f7a6440 100644
--- a/scripts/exyapps
+++ b/scripts/exyapps
@@ -1,4 +1,4 @@
-#!python
+#!/usr/bin/env python
 
 import exyapps.main
 exyapps.main.main()
diff --git a/setup.py b/setup.py
index dbe1551..4bbb3cc 100644
--- a/setup.py
+++ b/setup.py
@@ -7,33 +7,32 @@ from distutils.core import setup
 description = "Extensions of Yet Another Python Parser System"
 long_description = \
 """
-YAPPS is an easy to use parser generator that is written in Python and
-generates Python code.  There are several parser generator systems
-already available for Python, but this parser has different goals:
-Yapps is simple, very easy to use, and produces human-readable parsers.
+EXYAPPS is an easy to use parser generator that is written in Python and
+generates Python code.  It is intended to be simple, very easy to use,
+and produce human-readable parsers.
 
-It is not the fastest or most powerful parser.  Yapps is designed to be
-used when regular expressions are not enough and other parser systems
-are too much: situations where you might otherwise write your own
-recursive descent parser.
+It is not the fastest or most powerful parser.  Exyapps is designed
+to be used when regular expressions are not enough and other parser
+systems are too much: situations where you might otherwise write your
+own recursive descent parser.
 
-This package contains several upward-compatible enhancements to the
-original YAPPS source:
+Exyapps is derived from YAPPS, with various extensions:
 - Handle stacked input ("include files")
 - augmented ignore-able patterns (can parse multi-line C comments correctly)
 - better error reporting
 - read input incrementally
-
-Exyapps is an extended fork of yapps with these new features:
--  (to be written)
+- the generated parser does not require any runtime library
 
 """
 
 setup (
     name = "exyapps",
-    version = "3.0",
+    version = "3.0dev",
     description = description,
     long_description = long_description,
+    url="https://svn.stsci.edu/trac/ssb/etal/wiki/exyapps",
+    maintainer="Mark Sienkiewicz",
+    maintainer_email='no_spam@see_url',
     # bug: replace this and put acknowledgements of these guys in the docs
     # url = "http://theory.stanford.edu/~amitp/yapps/",
     # author = "Amit J. Patel",
@@ -45,5 +44,6 @@ setup (
     keywords = ['parsing'],
     packages = ['exyapps'],
     scripts = ['scripts/exyapps'],
-    #cmdclass = {'bdist_rpm': MyBDist_RPM},
+    # if we ever start using distribute
+    # zip_safe = False,
     )