Mime-proxy

Version 2.1.c.2,  July 19, 2006 Patrick Lamaizière  <patrick(dot)softs(at)lamaiziere(dot)net>


Purpose.

Mime-proxy is a program to convert characters sets in news articles. It adds a support of characters sets to newsreaders (as Xnews or Gravity) that doesn't managed them.

Outgoing articles ( from the client to the server ) are encoded to the appropriate characters set. Incoming articles are decoded to displayed them. the application is a proxy server, so it relay posts from news servers to one or more newsreaders posts. To encode properly outgoing mails, a SMTP proxy is also available.

Features.

    Support characters sets via the library libiconv
    Encode / decode quoted-printable encoding.
    Encode / decode base 64 encoding.
    Encode / decode encoded words (RFC2047).
    Manage multipart MIME articles.
    Support yEnc encoding
    Multi servers, one configuration for each.
    Possibility to set configuration via article's header.
    Protocols : NNTP, POP, SMTP.
    Operating Systems  : Windows ( 98, Me, NT4.0, 2000, XP )  / POSIX

Encoding .

The program encode the body of the article by trying each characters set defined in the configuration file. When a set matches all characters found in the post, MP converts the article from the newsreader's characters set to this characters set. If it can't encode the article, MP rejects it.

MP can encode the body to quoted-printable and headers to "encoded-words" RFC2047. If the newsreader uses itself encoded-words, the proxy uses the client_charset parameter [see configuration] as alternative charset.
 
Mime-proxy adds the following headers:
Mime-Version: 1.0
Content-Type: text/plain; charset='used charset'.
Content-Transfer-Encoding: 7bit,8bit or quoted-printable.

Mime-proxy will replace those fields in the original article if needed. Mime-proxy uses a Unicode reference map (16 bits), so a non-Unicode characters can't be encoded or decoded. Last, this software will modify the 'User-Agent' field for self-advertising.

Decoding.
   
Posts that indicate MIME encoding are decoded to the character sets used by the newsreader. So it will be well displayed. Encodeds word, quoted-printable and utf-8 are also decoded.

Install.

This software consists of an executable and only need the library libiconv (you will find a DLL on my web site) and a configuration file to run :

Extract those files from the archive into a chosen directory  and edit the '.ini' file (see configuration for more).
Put the libiconv dll into Mime-proxy's directory.
  
Uninstall

Delete previous files :-)

Running
   
From the shell window :

mproxy.exe configuration_file [Options]

where : 
    configuration_file : .ini like file for settings (see configuration for more)

Options

-silent : Do not display anything, use the task manager to watch the application activity.
-free_console : Don't use any console, use the task manager to watch the application activity.
-low_memory : Garbage collector management : safe memory regardless to CPU activity.
-default_memory : Default Garbage collector management : somewhere between low_memory and high_memory
-high_memory : Garbage collector management : not safe memory.
-lint   : Display the configuration and exit.
-user username : Username to switch after the installation of the servers; to avoid running MP as root (under Unix only).
The more simple way to launch the soft is to create a shortcut under Windows.
configuration_file can use path, If no path is specified, Mime-proxy looks for the file into the current working directory and after into the Mime-proxy's directory.

Configuration.

The configuration file is a text file, as common windows .ini files. You can check this file with the "-lint" option, in this case Mime-proxy displays the configuration and exit :

mproxy.exe mproxy.ini -lint

Edit the « mproxy.ini » file and adapt it according your connections settings, charsets you want to uses and options.

Settings in the standart mproxy.ini file are adapted for a newsreader that uses windows-1252 characters set and for the fr.* hierachy (no encoded-word, 8 bits encoding) . You may be should adapt it.

For fr.*, just change settings for connections in the section [proxies]

Proxies configuration.

This section describes connection parameters for each proxie :

[proxies]
server_alias = local_port_number, remote_host, remote_port_number [,protocol]

Where server_alias is a name for identify this proxy in the configuration file.
local_port_number : the local port to listen.
remote_host : the remote host to connect to.
remote_port_number : the port on the remote host to connect to.
protocol : specify the protocol (nntp, pop, smtp or incoming_smtp or nonr). This parameter is optional. Default is nntp.

The "none" protocol do nothing, in this case MP acts as a simple proxy.

Example :

[proxies]
hamster = 6000, localhost, 119
free = 6001, news.free.fr, 119
wanadoo = 6002, news.wanadoo.fr, 119, nntp
smtp_wanadoo = 25, smtp.wanadoo.fr, 25, smtp
pop_wanadoo = 110, pop.wanadoo.fr, 110, pop

The proxy listening on port 6000 connects to the Hamster server, the one listening on port 6001 connects to the Free news server, the one listening on port 6002 connects to the Wanadoo news server. The one listening on port 25 is an smtp proxy connects to the Wanadoo outgoing mail server. The last one is a pop proxy.

Advanced configuration.

You can define a different configuration for each proxy. If no configuration section is defined for a proxy, the [default] section is used. To associate a configuration to one proxy, create an section named as the server_alias used in the [proxies] section.

Example :

[proxies]
hamster = 6000, localhost, 119
free = 6001, news.free.fr, 119

[default]
encoded_word = 1
charsets = us-ascii, iso-8859-1, iso-8859-15, utf-8

[hamster]
encoded_word = 0

[free]
encoded_word = 1

In this case, the proxy will used encoded words on the free server but not on the hamster server. By the way, the free section define the same value as the default section, so this one is not really needed.

Characters sets.

Mime-proxy handles many charsets (see http://www.gnu.org/software/libiconv ). Mime-proxy has also support for transliteration, i.e. when a character cannot be represented in the target charset, it can be approximated through one or several similarly looking characters. Transliteration is activated when "//TRANSLIT" is appended to the charset. By sample, the euro symbol can be approximated by "EUR" in the iso-8859-1//TRANSLIT charset.

Parameters.

o charsets = charset1, charset2, ..., charsetn

Specify the charsets to use for encoding. Mime-proxy try the first defined charset then the followings until it finds a charset matches all the characters of the message. If no charset matches, Mime-proxy rejects the article.

Example :

charsets = us-ascii, iso-8859-1, iso-8859-15, utf-8

o header_charset = charset

Specify the charset to use when encoding "raw 8 bits header." If this parameter is not set, MP uses the "client_charset" parameter as charset.
In case of RFC2047 encoding (parameter encoded_word=1), this parameter is not used.

Exampe :

header_charset = iso-8859-1

o client_charset = charset

The charset used by the client (the newsreader) for outgoing articles. MP translates outgoing articles (from client to server) from this charset.

Example :

client_decoding_charset = windows-1252

o trust_client_charset = 0 | 1

If set, Mime-proxy uses the characters sets of outgoing articles instead of  the "client_charset" parameter. Default value 0.

o header_charset = charset

Specify the charset to use for encoding of the header in case of raw 8 bits header.

header_charset = iso-8859-1

o client_decoding_charset = charset

The charset used by the client (the newsreader) for incoming articles. MP translates incoming articles (from server to client) to this charset. If this parameter is not set, Mime-proxy uses the client_charset parameter.

Example :

client_decoding_charset = windows-1252

o decode_header = 0 | 1

Enable (1) / disable (0) the decoding of the header in incoming article. with the "decode_header" parameter. Default value : 1.

o enable_mine_proxy_header = 0 | 1

Enable (1) / disable (0) the "X-Mime-proxy:" header. Default value : 1

o encoding = 0 | 1

Disallow (0) / allow (1) all encoding features. Default value : 1

o encoded_word = 0 | 1

Disallow (0) / allow (1) the use of encoded words (RFC2047) in outgoing articles. Default value 0

o incoming_encoded_word = 0 | 1

Disallow (0) / allow (1) the use of encoded words (RFC2047) in incoming articles. Default value 0. If this parameter is set, Mime-proxy encodes XOVER and incoming articles' headers into encoded-word.

o encode_to_qp = 0 | 1

Disallow (0) / allow (1) the quoted-printable encoding for outgoing articles. Default value 0

o encode_to_b64 = 0 | 1

Disallow (0) / allow (1) the base 64 encoding for outgoing articles. Default value 0

o decoding = 0 | 1

Disallow (0) / allow (1) the all decoding features. Default value 1

o enable_incoming_yEncode = 0 | 1

Disable  (0) / Enable (1) support for yEnc in incoming articles.

o enable_outgoing_yEncode = 0 | 1

Disable  (0) / Enable (1) support for yEnc in outgoing articles.

o xover_decoding = 0 | 1

Disallow (0) / allow (1) the decoding of headers in XOVER command. Default value 1

o unknown_character : one_character

The character to use to replace any non recognised character. Default value ?
( sample : unknown_character = % )

o ip= ip | ip1-ip2 [,ip | ip1-ip2]

client ip access control : use ip to allow a single ip (as 127.0.0.1) or ip1-ip2 to allow an ip range :
sample :
    ip = 127.0.0.1, 192.168.0.1-192.168.0.16

Default value : 127.0.0.1

o message_id =  format_string

MP can generate a message-id field. It uses the format_string, any character '#' followed by two digits will be replaced by a random hexadecimal number with the number of digits.

sample:

    message_id: Xns#13totoplam#02@news.free.fr
    Should generate a Message-ID: field like :
    Message-ID: <Xns0123456789ABCtotoplam01@news.free.fr>

Warning: do no append '<' and '>' in the format string.
Default value : none

o date = 0 | 1

MP can generate a Date field (1) or no (0). Default 0

o default_charset = charset

If no charset is specified in the Content-Type field, MP uses this charset as default charset. This parameter applies to headers and body.

sample :
    default_charset = iso-8859-15

o original_ip_header = 0 | 1

If set, MP adds a X-Original-IP: header filled with the address IP of the client. Default value 0

o original_ip_reverse = 0 | 1

If set, the "X-Original-IP:" header is filled with the reverse dns of the address IP of the client. Default value 0

o replace_characters = character_to_search - character_to_put

"search and replace characters" feature for outgoing article. Characters are specified by the unicode value in hexadecimal.

By sample to replace the character RIGHT SINGLE QUOTATION MARK (Unicode 0x2019) by the character APOSTROPHE (Unicode 0x27) :

    replace_characters = 0x2019 - 0x27

You can make more than one replacement by separate them with a coma :

    replace_characters = 0x2019 - 0x27, 0x.... - 0x ....

Replacements can also be specified with the header 'X-Mp-Replace:'
X-Mp-Replace: 0x2019 - 0x27


o smtp_helo_to_ehlo = 0 | 1

If set, Mime-proxy translates the 'HELO'  command (SMTP) by 'EHLO'.  This is a hack for Xnews. Default value 0.

Other parameters.

The [misc] section allows to tune other Mime-proxy's parameters :

o socket_buffer_capacity = integer

Set the capacity of the sockets' buffer. The default capacity is 2048 bytes. If you've got some problems when sending big article, try to decrease the cpacity of this buffer.

      socket_buffer_capacity = 1024

o fork_connection = 0 | 1
 
Under *nix, Mime-proxy can fork each connection.
    fork_connection = 1 (default 0)
 
If fork failed, MP rejects the connection with an error "No more ressources available ! Try later"

o max_child = integer
 
Check the number of forked child. If the number of child is greater than this number, connection is rejected with an error "No more ressources available ! Try later"

    max_child = 25 (default 50)

o max_managed_sockets = integer

Check the number of managed sockets.  MP rejects the connection if the number of managed sockets is greater than this parameter with an error "No more ressources available ! Try later".

    max_managed_sockets = 200 (default 128)

MP needs one socket by local server and two sockets by connection to handle the connection between the client and the remote server. If connections are forked, only local server sockets are counted.

Special headers.

Encoding parameters may be set from the original post by initializing the
following fields :

Encode outgoing article ( from client to server )
    X-Mp-Encoding: no/yes

Append to User-Agent:
    X-Mp-User-Agent: no/yes

Use encoded words :
    X-Mp-Encoded-Word: no/yes

Encode to quoted-printable:
    X-Mp-Encode-To-Qp: no/yes

Encode to base 64:
    X-Mp-Encode-To-B64: no/yes

Encoding Charset :
   X-Mp-Charsets:  First,Next,...Last charset

Encoding charset for raw 8 bits header.
    X-Mp-Header-Charset: charset

Support for yEnc encoding :
    X-Mp-yEncode: no/yes

Search an replacement feature :
    X-Mp-Replace: 0x2019 - 0x27

 Trust client charset:
    X-Mp-Trust-Client: no/yes


Those fields will be removed from the post before sending to the server.

Those headers could be disallowed by setting the option:
    allow_mp_control_header = 0 | 1 (default : 1 )

Language.

You can change application's user messages, see mproxy_en.dat file for a translation in english. Edit this file and rename it in mproxy.dat if you want to use it. Default language is french.

Newsreader settings.

You should change connection settings in your newsreader. Use 'localhost' for the server name and as port number the port that MP is listening. Keep other parameters as password, login, ...

It doesn't work.

MP uses Winsock 2.0.If you uses Win95 you must get this dll from Microsoft.

You should have a display as :
«
Mime-proxy version: 2.0.c.0

Installed on port: 5000, localhost, 119, nntp
Installed on port: 5001, news.free.fr, 119, nntp
Installed on port: 5002, smtp.wanadoo.fr, 25, smtp
»

I can't connect to the proxy.

Check your newsreader parameters and firewall parameters if any.

Thanks.


David.

David Epelbaum.
Piotrek.
Luca.

SmartEiffel : http://SmartEiffel.loria.fr
Elj :  http://www.elj.com/elj-win32/
Eposix : http://berend.gameren.nl/eposix/
Gobo : http://elj.sourceforge.net/projects/other/gobo/
Nenie :  http://sourceforge.net/projects/nenie/
Ucstring :  http://sourceforge.net/projects/ucstring/
Yaesockets : http://sourceforge.net/projects/yaesockets/
Libiconv : http://www.gnu.org/software/libiconv/

Licence (file forum.txt).

The library Libiconv  is available under the LGPL license, see COPYING.LIB.

-----------------------------------------------------------------------------------------------
 Mime-proxy.
 
 Copyright (c) 2002-2006 Patrick Lamaizière <patrick(dot).softs(at)lamaiziere(dot)net>
 -----------------------------------------------------------------------------------------------
 
Eiffel Forum Freeware License, version 1

Permission is hereby granted, without written agreement and without
license or royalty fees, to use, copy, modify and/or distribute this
package, provided that:

  - copyright notices are retained unchanged

  - any distribution of this package, whether modified or not,
    includes this file

Permission is hereby also granted, without written agreement and
without license or royalty fees, to distribute binary programs which
depend on this package, provided that:

  - if the binary program depends on a modified version of this
    package, you must publicly release the modified version of this
    package - for example by submitting it to the Eiffel Forum archive
    (http://www.eiffel-forum.org/archive/ )

THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT WARRANTY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,BUT NOT LIMITED TO, THE IMPLIED  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS BE LIABLE TO ANY PARTY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THIS PACKAGE