Python 

Oblivio

Sinopsys

Oblivio is a tool for index generating of a directory tree, for html-like pages it also built the TOC.
By default if none root directory is specified it assume root is the python os.curdir (if you lounch it from file manager it's the dir where you click it, else it's the shell current dir.)
It's only output for the moment is XML; that needs to be cleaned by Tidy only if your file or directory names involve the use of xml entities, Oblivio actually doesn't check it, the script calls tidy automatically, but it must be configured properly.
The XML looks like the dirs tree; specifying file and dirs name, if you wan't you can specifie more file pattern, actually the script is able to parse (X)HTML-like, mp3, and .py files, using other pattern Oblivio will return only basical infos like size, last modifed date, and so on.
See the Example: Site Map
Obviously you need of other transformation to make your map useful, I use a very simple XSL stylesheet that transform the XML to XHTML. You can exlude directories, directory trees and files using also regex (or simple you can use an * as in the command line; it stands for an indeterminated number of characters), the program ,also, check empty directories (it checks if specified files are in the dir), to exclude them.
At the end of the map there's a statistics area for more infos about elaboration process. Be aware, time section is strictly the time needed for pages elaboration not for the entire works such as importing modules, writing output and so on, but it isn't difficult to write a script counting all the needed time.

Configuration parameters:

{



'skipDirs':(r'/image',r'/cvs',r'/*_files', r'/Orig_templates')



'excludeFiles':(r'/report.html',r'/LinkVer_report.html')



'filExt':('pdf','htm','html')



'excludeEmpty':1



'timeFormat':"%d %b %y, %H:%M:%S"



'stat':1



'openMap':0



'readTidyOutput':1



'starTidy':1



'tidy':(r'tidy',os.curdir + r'/Orig_templates/xml.tidy')



'output1line':'<?xml version="1.0"?>



<?xml-stylesheet type="text/xml" href="xx.xml"?>



<root>'



'plugIns_dir':r'\Obl_plug-ins'



'root': os.pardir



'dest':'mappe/map.xml'



'FullCnt':1



}

Times and statistics

Calling the script via an other module: 11 secs for 105.531KB (version 0.17)

Source

Oblivio0.17
XX.xsl
xml.tidy

Archives

Oblivio0.1.py
Oblivio0.11.py
Oblivio0.12a.py
Oblivio014.py (not stable yet, don't download it!!!!)

Ideas for next releases

• Char Encoding and decoding from Html Entity
  
• Oblivio needs to be more extensible, so I'm planning a new way to call file elaboration classes
  Done Actually the extension system is based upon independent classes inside the main source file, since next release plug-ins will be searched in other dirs.
• Configuration via an external file.
  
• Not only HTML based file, but it may be useful to index also other files, such as *.doc, *.swx, *.pdf,...
  Done Actually Oblivio is able to elaborate all sort of file, it provides more informations for *.html, *.mp3, if you have suggestion for *.pdf, please, write@me
• Thanks to the H[1-6] based elaboration final output encreases too much. It can divide output in more files linked togheter.

Change log

##VERSION 0.17

##8 Mar 2005
Improved pythonic code.
File and directory exclusion works with regex

##VERSION 0.16

##28 Nov 2004
Finally a new decent release!

• Improved velocity.
• Now you can buiild your own extension very easely.
• Simpler way to configure the program,
• You can specify a list to file pattern, if Oblivio has extensions to elaborate them it build the map with them, else it put out few infos such as size, last modificated date, and so on. If you need a special analisys you can build your own.
  

##VERSION 0.12b

##4 Oct 2004
Version 0.12 will be released only with a working and stable configuration.

• Files exclusion
• Directories exclusion
• Less redundant code, improved speed
• Now it can be both stand alone script and module
  
• Preparing for a simpler configuration
• • Files extension all in a single list
  • Root directory also relative to os.getcwd() or auto-detected

##VERSION 0.11

##29 Sept 2004

• Insert statistics:
• • elaboration time
  • max file size
  • min file size
  • medium file size
  • total file size
  • bytes per second
• created a basic XSL stylesheet for XML output. Link to these stylesheet is inserted by default in the output.
• Fixed bug in class Page: relUrl returned a bad value. Now it's correct.
  

##VERSION 0.1

##26 Sept 2004

• Define a root directory
• Elaboration of predefined files
• Auto-exclusion of empty directories
• Extraction of informations directly from the page:
• • Description
  • Tile
  • Last modified
  • Size
  • H[1-6]
  • Relative to root URL