Tuesday, June 12, 2007

How-To: Image Windows XP with Ghost and Sysprep (Part 3)

Source: http://blog.hishamrana.com/2006/02/22/how-to-image-windows-xp-with-ghost-and-sysprep/3/


Update (2.24.2006): I’ve gotten some great tips from both the comments at Digg and below. Let me address a few of those.

  • Once Sysprep shuts down the computer, if you are able hook up the drive to another system as a secondary device then delete pagefile.sys and hibernat.sys to save a few hundred megs of space in the final image. Thanks master_of_fm.
  • I forgot to go in-depth on the Ghost process. I’ve included basic steps for Ghost 2003 but once the intial Sysprep deployment is made, any disk imaging program can be used. The basic priciples are the same but let me know if you have problems. Thanks hakujin & definiteform.
  • billyboobs34 asked if it is possible to make an image that is totally hardware independant. The quick answer (to be best of my knowledge) is no. The problem hinges on the HAL (Hardware Abstraction Layer) in Windows XP being different on, for example, a uniprocessor and multiprocessor system. The Microsoft Knowledge Base article is linked above. If the HAL is not the problem then it sounds like there may be more of an issue of finding appropriate chipset and mass storage controller drivers for different motherboards. In the custom drivers section, adding in paths to the necessary drivers for other motherboards may mitigate this problem.

    In my opinion, the best solution is to have one basic machine for each HAL to make all of your images from. Follow all the steps above but do not reseal the system. Drop down to Ghost (or whatever backup utility you are using) and create an image of the system. In the future, creating compatible images for newer machines would be as simple is restoring the almost-complete Sysprep image and adding in new drivers for the chipset, mass storage controller and anything else you need. Then resealing the system and imaging it for a viable deployment image. Again, I am not an expert so hopefully the Sysprep gurus can step in to address this one if I went wrong somewhere.

  • If you are presented a blue screen of death after restoring the Sysprep image to a target machine, then chances are that either the HAL is incompatible for the target machine, you forgot to build the mass storage driver section or you forgot to add a third-party mass storage driver. Refer to Microsoft KB309283 to confirm the source machine HAL is compatible with the target machine. Refer to Microsoft KB303786 if a Stop 0×7b blue screen is shown. This means that the mass storage driver section or third-party mass storage driver is missing.

Update (3.8.2006): If after rebooting, the install hangs at mup.sys or agp440.sys then it’s more than likely a HAL problem. Refer to the previous update to change your HAL. For most modern day single-processor machines make sure your HAL is changed from Uniprocessor ACPI to Advanced Configuration and Power Interface (ACPI) PC. You can find this setting in Control Panel -> System -> Hardware tab -> Device Manager -> click (+) next Computer -> right click the HAL shown and pick Update Driver -> Install from a list or specific location (Advanced) -> Don’t search. I will choose the driver to install. -> then pick the proper HAL -> hit next/okay until you finish. Now reboot and run through the Sysprep instructions again.

It took me a few tries to finally make an image that restored properly when I first started but, like riding a bike, it’s second nature now. Be ready to experience frustration but in the end it is worth having this knowledge. Again, read the comments below and at Digg to see how others have achieved this process. I don’t claim to be an expert (I’m just a med student by day) but I certainly hope this helps someone.

How-To: Image Windows XP with Ghost and Sysprep (Part 2)

Source:
http://blog.hishamrana.com/2006/02/22/how-to-image-windows-xp-with-ghost-and-sysprep/2/

Part 2: Norton Ghost Instructions

I’m using an older version of Ghost because the newer versions are trumped by Acronis True Image in terms of having less bugs and more features. However, DOS based Ghost has worked fine for me for most systems. If Ghost fails on your system for some reason, try imaging with Acronis True Image trial version. The basic idea is boot off a floopy or CD on the source system, start the backup program, backup the entire drive as an image file onto some other device (portable hard drive or another drive in the system) and then restoring (as necessary) from that image file onto a target system.

The basic Ghost process is as follows:

Note: You must do this on another machine that has Ghost installed. If you boot the Sysprep machine accidentally then you must go through the entire Sysprep checklist again. Trust me, I learned the hard way more than once.

  1. A Ghost bootdisk must be created for the target system. In Ghost 2003, start Ghost from the Start Menu -> Programs -> Norton Ghost.
  2. Click Ghost Utilities then click Norton Ghost Boot Wizard.
  3. A wizard comes up. Since I plan on saving my image to a portable USB or Firewire hard drive I pick Standard Ghost Boot Disk. You may have to choose another option if you plan on saving the image to a network Ghost image server.
  4. On the next screen, under External Storage Options click the check boxes for USB 2.0 Support or Firewire Support depending on your target backup device. If you are using an internal secondary hard drive, then you don’t need to pick anything.
  5. On the next screen, click Use PC-DOS.
  6. The next screen asks for the path of ghost.exe. The default path is usually fine.
  7. Finally we are presented with a screen asking for the floppy disk drive letter and formating options. Pick the proper drive letter and leave the format options alone. Click next.
  8. This is the Review screen. Make sure all the options are correct and insert an empty floppy into your drive. A format dialog will pop up with a warning that everything on the floppy will be erased. Hit Okay. Close it once it is completed. Ghost will copy the files over.
  9. If you are using an external hard drive then make sure it is connected to the source machine prior to booting the Ghost floppy on the system. If you are using an internal hard drive (not the same one as the Sysprep’d drive) in the source system then make sure there is enough free space on it. Again, you cannot save the image onto the drive/partition you are imaging for obvious reasons.
  10. Put your shiny new floppy into the Sysprep prepared machine and boot off of it.
  11. Create the image following the on-screen directions.
    1. Click Local.
    2. Click Disk. I’ve assumed that we are imaging a single large NTFS partition encompassing the entire drive throughout this tutorial hence the need for either an external drive or different internal hard drive. This can all be done on a single drive with two paritions but I’ll leave for the end user to explore.
    3. Click To Image. The next screen states Select local source drive by clicking on the drive number. Make sure you select the drive that contains the Sysprep prepared XP deployment! Check and double check this. Click Ok.
    4. The next screen reads File name to copy image to. From the drop-down box pick either the external hard drive or internal drive (different from source drive) and then type a short name in the file name box.
    5. The next screen will ask compression options. I usually pick High.
    6. Ghost will image the drive on its own and alert you when it is done.
  12. If you are sure the image was created successfully, you can now move the image onto a burnable DVD, network share, leave it on the portable or secondary hard drive and restore it as necessary. If there was a problem during the imaging process, then do not boot the source system. Just reimage it after addressing whatever caused the image process to fail.

The steps above are a very quick summary so I recommend that anyone truly interested in doing this to head over to Vernalex’s site for an in-depth Sysprep Guide. Along with a lot more detail of the entire process, he has various tools to make everything a little easier. The best utility on his site has to be the Sysprep Driver Sanner tool. It makes the process of adding hardware drivers much easier than the manual process above. Instructions on using it are on his site.

This completes the imaging process. Part 3 includes miscellaneous notes.

How-To: Image Windows XP with Ghost and Sysprep (Part 1)

Source: http://blog.hishamrana.com/2006/02/22/how-to-image-windows-xp-with-ghost-and-sysprep/

I tend to install and reinstall operating systems quite frequently on my home system. With my hectic schedule, I don’t have time to sit down to reinstall and configure every last program. Although it is arguably easier to use a backup of my system for day-to-day mishaps, I tend to change out peripherals a lot and restoring a system that does not have certain drivers “cleaned out” tends to wreak havoc on a new configuration. To make this process go a bit faster, two years ago I created a “Ghost Image” of my hard drive after doing a basic install. Now, instead of it taking a few hours of toiling away to reinstall a system, I can do this all in about 15 minutes with only half a dozen quick mouse clicks. The trick is to use Norton Ghost or any other imaging software and Microsoft Sysprep.

Part 1: Install XP and Sysprep

  1. Install Windows XP on a clean hard drive.
  2. Do not install any drivers or other utilities that are hardware specific beyond what Windows itself installs.
    • This is necessary to make sure the image is as portable as possible across different types of systems. However, different storage controllers and different HALs (Hardware Abstraction Layers) make this harder to predict.
    • Most modern computers these days work fine with a standard ACPI HAL, but if this image is to be truly portable across multiple machines then it must be determined which specific HAL will be needed. Refer to Microsoft KB309283 if you are completely lost.
    • It is also important to determine if the target system uses a storage controller that normally requires a driver disc during a regular XP install. If this is the case, then the necessary paths to the drivers must be included in the Sysprep.inf file. These must be added to the [SysprepMassStorage] section in the form PCI\VEN_###&DEV_#### = PATH_TO_DRIVER_ON_IMAGED_DRIVE where VEN_#### should be replaced by the Vendor ID number (i.e. VEN_1234) and the DEV_#### should be replaced by the Device ID number (DEV_1234). This information can usually be found in the specifc driver INF files. Here is an example for adding the VMWare SCSI controller driver to sysprep.inf

      [SysprepMassStorage]
      ….snipped out windows mass storage driver list….

      PCI\VEN_104B&DEV_1040=C:\Drivers\Mass\VMWare\vmscsi.inf

  3. Create a testuser account with administrative privileges. Use this account to install and configure all the software and policies on the system.
  4. Remember to run Windows Update, Office Update and make sure all the rest of the software is up to date. You’ll probably end up rebooting a few times in between but keep going until everything is updated.
  5. Copy all the start menu items from the testuser account to the Administrator start menu. (Note: This is necessary as some installers do not create start menu items in All Users but within the testuser profile only. This leaves some items missing on the Administrator start menu.)
  6. Log out and log back in as the computer Administrator and then copy the testuser profile folder to the default user profile folder. This is done via Control Panel -> System -> Advanced -> User Profile “Settings” then select testuser and click Copy to. Copy all of this to c:\Documents and Settings\Default User. If you don’t understand then refer to Microsoft KB291586.
  7. Delete the testuser account. Make sure that c:\Documents and Settings\testuser has been deleted too.
  8. Download Sysprep for XP SP2.
  9. Extract the files to c:\sysprep.
  10. Create the basic sysprep.inf file by running setupmgr.exe. This a tool Microsoft provides for creating an answer file so the restore doesn’t involving asking the normal setup questions. The basic steps are below:
    • Run setupmgr.exe
    • Click Create New
    • Click Sysprep Setup
    • Then choose whichever product you are using. In our example it would be XP Professional.
    • The next question asks: Do you want to fully automate the install? All this question determines is who is going to accept the EULA, you or the person restoring the image. Also, picking yes means that you must enter your Product Key. I pick no because this is for my own use and I don’t want someone to swipe my Product Key accidentally, but a large company or OEM may choose differently.
    • The next few sets of options are for you to enter in any information like your Name, Organization, Time Zone, Product Key (I leave this blank), Network Settings, etc.
    • I leave the Computer Name option set to Automatically generate computer name.
    • Once completed, a dialog box will ask where you want to save the file. c:\sysprep\sysprep.inf is the path we’re using in this example.
    • On the completion screen, click Cancel to close setupmgr.exe.

    The process of creating a basic sysprep.inf file is now completed.

  11. Before proceeding to the next step, create a custom hardware drivers directory for any drivers needed for the target system. Usually I use c:\drivers.
  12. Open c:\sysprep\sysprep.inf in Notepad and add the following lines to the relavent sections (if the heading doesn’t exist, create it):

    [Unattended]
    DriverSigningPolicy=Ignore
    UpdateInstalledDrivers=Yes
    OemPNPDriversPath=drivers\hardware_cat\driver_dir\driver_inf;(repeat);

    [SysPrep]
    BuildMassStorageSection=Yes

    [SysprepMassStorage]

  13. Do not close the sysprep.inf yet! OemPNPDriversPath points to the c:\drivers directory created earlier. For organizational purposes, I split up my custom driver files based on category (i.e. hardware_cat in the example above). For example, all video drivers go under c:\drivers\video and network drivers under c:\drivers\network. In each of those directories, the specific driver bundles are placed with their driver inf files (i.e. driver_dir). For example, the latest nVidia drivers would go into c:\drivers\video\nVidia\. The last part is refering to driver_inf is just that, the name of the inf file. For example, for the latest nVidia driver, the path would be c:\drivers\video\nVidia\nv4_disp.inf. In sysprep.inf, the path would be written as OemPNPDriversPath=drivers\video\nVidia\nv4_disp.inf;. Do not forget the semi-colon as a separator. For the next driver, repeat the procedure by placing the path after the semi-colon without leaving a space. Once all the drivers are added, save the file.
  14. Run c:\sysprep\sysprep -bmsd. This will build the Windows XP standard mass storage drivers section.
  15. While editing sysprep.inf there is an option labeled InstallFilesPath which usually points to c:\sysprep\i386. I usually copy the contents of my XP CD’s i386 directory into c:\sysprep\i386. This isn’t necessary.
  16. Add any custom Storage dirvers to the [SysprepMassStorage] section as detailed above.
  17. Now run C:\sysprep\sysprep.exe.
  18. Pick options Mini Setup and Detect non-plug and play hardware. If you don’t have a volume license and plan on just using this image for restoring the computer the image was made on, then pick the option Don’t regenerate security identifiers. If you have a volume license key and will be using this image for multiple machines then leave that option unchecked. Ensure that Shutdown is selected from the Shutdown mode drop-down menu and click Reseal.
  19. If you left the SID option to regenerate, then a pop-up will ask you to confirm. hit OK to continue.
  20. This will take a while and your system will shut down once the process is complete.

Sysprep is now complete. Part 2 contains instructions on imaging the partition.

Thursday, June 7, 2007

My favorite Mambots

Source: http://www.eyezberg.com/index.php?option=content&task=view&id=90

1. Run Digital

  1. Read More

    A Mambot to display a Read More link under your content item.
    The advantage is that this link can now be displayed before commenting.
    The Bot uses existing menu links to determine the Itemid (responsable for highlighting the active menu item for example).
    Furthermore, it uses the title-alias as linktext if present, else the item title will be used.
    No tag needed.

  2. Previous / Next Link

    This Bot replaces the normal "Content Item Navigation" by its own interpretation.
    The Texts "Prev" and "Next" are replaced by the "Title" of the items.
    The Bot uses existing menu links to determine the Itemid (responsable for highlighting the active menu item for example).
    The text of the links and the introduction header can by customized via Language-Files; currently these exist for German, English and French.
    Please disable the normal Content Item Navigation under Global Configuration/ Content or navigation would be shown twice!
    No tag needed.

  3. Add PHP

    Bot allowing to call any PHP script stored under your root directory into any content item.
    Please disable Print and PDF links for these items.
    Syntax: { rdaddphp file=rel_path_to_file_including_filename}

  4. Link Content

    Usefull to add links between your content items easily, as you use either the Titel, Title-Alias or the ID and specify the linktext to be shown.
    Syntax: { rdlc item=title or title_alias or ID of the itme | [&alt=alternative Text for the Link]}

2. Joomla Expert

  1. MosAuthorXTD 1.5

    This Mambot lets you create an individual string for the display of author, created, modified, usertype, section, category and hits of your content items.
    You can assign a CSS-class and display 4 different strings in 4 different positions at the same time.
  2. MosClearFulltext

    Gets rid of the "Read more" link on blog-pages, when there is no content in the fulltext (as some ditors insert a breakrow tag by default which is not visible in WYSIWYG mode).
    No tag needed.
  3. MosImageLink

    With this mambot, every first picture found in an introtext is linked to the full article (if fulltext is not empty).
    MosImageLink is NO replacement of the original "mosimage"!
    You don't need to edit any existing content to make this mambot work. Keep on inserting images like you did before.
    MosImageLink searches for the first occurence of the "mosimage" tag or an HTML-Tag starting with "If either one is found, MosImageLink adds a link to the article (but only if there really is a fulltext).
    No tag needed.

3. Globodigital

  1. Page Options

    This mambot not only allows you to use the print, pdf and e-mail icons but also the use of a bookmark site and bookmark page.
  2. Tips

    The Tips mambot allows you to make use of a highlighted area in your content and was designed to show tips and/or notes.
  3. My Vote

    This mambot shows a vote form similar to the core one but by using a mambot, you can place it anywhere in your pages.

4. Q-Square

  1. Amazon Associates

    With this mambot you can display an Amazon Associates advertisement in your content for a specific product.
    You can find the product id by searching amazon.com for a product and looking at the url.
    Make sure that you set your Associates ID or else it will use the developers Associates ID.
    Syntax: { amazon id='########' align='####'}

5. Kreacom

  1. MosIf

    When you publish content in Joomla, you can only choose what kinds of user are allowed to see the whole content item:
    • public (ie. all users),
    • registered (all logged-in users) or
    • special (like administrators).
    While this is often exactly what you need, it is an all or nothing choice.
    In contrast, MosIf allows you to control precisely what parts of a document people will see!
    Testable conditions are:
    • username
    • usertype
    • frontend language
    • IP adress
    • Cookies
    and tests can use either the = (equal) or the != (not equal) operators.
    Syntax: { mosif username=john,batman,whirleybird}Hello, nice to see you again!{else}Hi stranger!{endif}
  2. Mosval

    a new mambot that can display php variables and expressions in normal content.
    Syntax: { mosval "Hello. I am $my->username and I am a $my->usertype!"}
  3. Moshide

    Moshide is very usefull to show or hide content dynamically, for example on long pages, be it text or images.
    It can be used in (default) block mode, or tag mode.
    Notes: hidden content will be visible on printed pages!
    This bot uses no end tag!
    Syntax: { moshide[.] [hidden] |}{/moshide[.]}
  4. Htmlfix

    Simply wrap your HTML or javascript sections between {htmlfix} and {/htmlfix} tags directly in the wysiwig editor. When the document is displayed, the mambot decodes the HTML entities introduced by the editor so the code will work again.
    Syntax: { htmlfix} code here {/htmlfix}

6. D4J

  1. D4J MOS Image Extended Mambot

    MIE allows you to control the size of images presented in your Joomla content. You can set the image size threshold and every big image will be sized down and can be viewed in full size in the popup window. You can define the size for popup windows as well. Moreover, MIE utilize default Joomla "mosimage" syntax, so no changes has to be made in your current content.

7. Joomres

  1. MetaFly

    Automagic keyword generating bot with regular expression to remove anything within {} to avoid typing your meta keywords for each content item.
    No tag needed.

8. WikiBot

  1. This Bot integrates the successful opensource CMS Solution, Joomla, with the free Encyclopedia, WikiPedia. When activated [[wikitagged]] words in Joomla contents will become hyperlinked to WikiPedia articles.
    [wiktionary:, [[wikiquote:, [[wikisource:, [[wikinews:, [[wikibooks: -support; international (Fish) support, which links tagged words to correct wiki.language.site or a preferred one specified in the Bot config view.

9. Link Indication

  1. This content mambot inserts class="xxx" into each anchor tag according to the type of link:
    • external
    • external, link to wikipedia
    • external, ftp
    • mailto
    • internal
    You can configure the CSS classes to use from the bots parameters and set external link targets (new window), and use your own icons if you want.

10. Adsense

  1. Simple bot to display Google Ads in your content items.
    Don't forget to change the Ad Client value in the mambots parameters so it displays YOUR ads.
    You can choose to display Ads or AdLinks, Collapse empty ads or replace them with an alternate ad or color, set channels etc.. many configuration parameters when you edit this bot!
    Syntax: { mosgoogle [align]} where align can be left, right or center. Tag not requiered if set to auto.
    Remember: Google only displays a maximum of 3 Ads per page!


11. Joomlastuff

  1. sIFR

    sIFR allows you to use non web fonts on the internet while keeping your site accessible.
    This mambot that allows you to easily integrate this into Joomla for your titles.
    No tag needed.
  2. Overlib

    The overlib mambot makes it possible to use hover effects on text in Joomla.
    All you have to do is include the tag in your text, and make sure the mambot is installed:
    Syntax: { overlib linktext="text" text="text" title="popup title"}

13. DewPlayer

  1. Use this mambot to play your favorite mp3 right from your content items, with a very small Flash player.
    Place the files into the images/stories folder!
    Syntax: { play}your_file.mp3{/play}

14. Troozers

  1. Mosviewonly

    Bot which enables you to define which parts of your content are viewable by particular groups.
  2. Moshidefrom

    Bot which enables you to define which parts of your content are hiden from particular groups.

15. a6FlashMambot

  1. Easily adding Flash animations to your content.
    Syntax: { flash file="yourflash.swf" width="200" height= "60" align="right" play="true" loop="true" menu="false" }

16. DI-mediaViewer

  1. Bot designed to put language depended media files (image/flash) within content items.

17. Mos Document

  1. Easily add links to files and documents in your Joomla content using this mambot.
    It is probably best suited for those that do not need a full file-managment component, but still require some simple functionality.

18. Style Intro Text

  1. Wraps the Intro-Text of content items in a div styled with a CSS class, that is configurable.
    This allows you to style the intro text individualy, which currently it is not possible to do.

19. MOS Audience

  1. Show the visitor how challenging an article is, so the user can decide faster if the article matches their skills or not.

20. Moskeyword

  1. Simple mambot that applies a css class to all occurrences of a certain word(s) in content items.
    Useful for highlighting key terms in your content, for example have the company name always appear in a certain color.

21. Acronym Mambot

  1. This bot searches for well known acronym's like AKA, DVD or KDE for example and shows the corresponding description.
    The list of supported acronyms can be viewed / edited in the source code of the file acronyms.inc.

22. Josquote

  1. Creates graphically pleasing quotes from text in articles in an indented area you can style with CSS and images.
    Syntax: { josquote}quote something{/josquote}

23. jdownload

  1. Very simple mambot which allows you to add downloadlinks to your content. You can place the tags directly in your text.
    Create the directory "/images/stories/mosdl" and place the files you want to offer in this directory.
    Syntax: { jdownload filename.ext}Text for the Link{/jdownload}

24. Mini MP3 Player

  1. Play MP3 streams out of the content. The mini MP3 player is able to play and stop playing a song.
    Players can be modified in color and size by using additional parameters.
    MP3 files which are on an external server can be played using the included mambot external.

25. CorrectPNG !

  1. Fix the PNG transparency problem in IE 5.5+ and rollover issues with PNG pictures.
    No tag needed.

26. Fade Images

  1. Display several images in the same location and fade between them.

27. Googla Analytics

  1. Inserts the Google Analytics tracking code sitewide without having to edit your templates.

28. DeliciousBot

  1. Enables users to bookmark content items in http:// del.icio.us.
    When DeliciousBot is installed and published, it will place a link at the bottom of each content item to let the user bookmark that item in del.icio.us.

29. dynamicImage

  1. This is an enhanced { mosimage } mambot with lots of parameters like thumbnails, resize, popup(html, javascript or dhtml), thumbnail cache, image crop, fixed ratio, and more..

30. PlayerFLV Lite

  1. Simple plugin to insert a streaming Flash video file into your content items. This free version will only play one file, which needs to be named "video.flv" and be located in the /mambots/content/playerflv/ folder!
    Syntax: { playerflv}230|150|#000000|false{/playerflv}
    (parameters are width, height, background, autostart -I guess, as I haven't tested yet)

31. calDate

  1. Replace the standard Joomla dates for content items with cool little calendar page date items.
    See it in use on this site!
    You can even specify which sections, categories, or pages you would like them to appear on in the bot parameters.
    No tag needed.

32. TagBot

  1. Automate the creation of Technorati and Flickr tag by simply putting the word into double brackets ((like this)). The proper URL is generated. The plugin allows to specify flickr as an option this way ((flickr:Tag)), in this case the link points to the appropriate Flickr page. It also allows to choose a tag differing from the word it is linked to.

33. Michael Baas

  1. MB Append

    Use to append a constant (configureable) text to the article-output of Joomla articles. When this bot is triggered, the article itself has been displayed and any output you create now will be shown below the article.
    No tag needed, just publish and configure in the parameters the text (ex: breakrow tag) to append.
  2. MB LastDate

    Improves Joomla's display of article-dates by unifying Created and LastEdited dates into one convenient display (instead of above/below article). The bot displays one date only: the modified-date, if the article has been modified, date of creation otherwise. You can also specify separate texts that are append before and after the created and modified-date, and set the date format as you like. MB_LastDate honours Joomlas general settings, as well as individual settings for each article, and will only show the latest date if you have configured either generally, or specifically for the article, to show at least one date.(Unless you used the first config-parameter and selected to always run MBM_LastDate...)
    No Tag neede, just publish and configure.

34. Pollen-8

  1. Register Bot

    Display the registration form inside content.
    Syntax: { mosregister}
  2. Login Bot

    Display the login form inside content.
    Syntax: { moslogin}
  3. Content Bot

    Insert a piece of another Joomla! article directly into your current content item.

35. Mad Tooltips

  1. Easy layer- tooltip-creation which uses the overLIB library. Commands of the overLIB library can be triggered by passing arguments to the plugin syntax, or by defining default parameters in the backend parameters.
    Syntax: { tip [attributes]}

36. MOS Document

  1. Easily attach documents (zip, pdf, etc.) to a news item. In the bot parameters, you can configure the folder to use as default, and some additional settings which make using this bot very simple.
    Syntax: { mosdocument=filename.zip}Link Text{ /mosdocument}

37. Bot Podcast

  1. This bot features a nice configurable Flash player to integrate any podcast into your content easily.
    Skinable by editing parameters in the file "mambots/content/botpod/config.xml"
    Syntax: { pod}http://url_to_podcast.xml{ /pod}

38. aClips

  1. Drop text from one article into another. Add the specified number words of intro text from one article into the next by specifying the article ID and the amount of words in a the mambot parameters.
    Syntax: { aClips 5|30}

39. ClickBank

  1. Display relevant clickbank affiliate links.

40. MgMediaBot

  1. Diplay varius multimediafiles in your content or in a seperate poup. Supported for now rm, wma, wmv, mpg, avi, mp3, mov, mp4, flv and swf, and http, rtsp, mms links.
    Syntax: { mgmediabot}images/stories/video/videodatei.wmv|false|300|300{/mgmediabot}
    Parameters: {mgmediabot}[1]|[2]|[3]|[4]{/mgmediabot}
    1. your media file, supported for now rm, wma, wmv, mpg, avi, mp3, mov, mp4, flv and swf, you can use absolute, relative and full URLs, if you use a subfolder for your joomla Installation and like to display the media inline you have to use only relative or full URLs
    2. false for displaying inline, true(Link title) for displaying a link with "Link title" as linktext to popup
    3. width of displayarea/popup
    4. height of displayarea/popup
    Note: please read the enclosed readme!

41. mosflash

  1. Easily embbed Flash movies within content. Global defaults can be set through the administration screen in Joomla and be overriden within the mambot's parameters individually.
    Syntax: { mosflash file="filename.swf" width="100" height= "100" align="right" bgcolor="FFFFFF" flashbase="flash/ quality="high"}
    If you ommit the flashbase folder name then the images/stories folder from the Media Manager is used.

42. Review

  1. Include an abstract into a section or categorie page about a CD or movie for example.
    Syntax (example, incomplete): { mosreview}art=CD / jahr = 2004/preis= CHF 30.00{/mosreview}
    Note: check out the readme at the link, many parameters to set!

43.Mostoc

  1. Add an automated hierarchical table of contents to your content item. Mostoc creates anchors at your tags and links to them.

44. getdirections

  1. This mambot creates a small form with a textfield that can be used to get directions to a location. The form opens maps.google.com in a new window with the directions.
    Syntax: { getDirections destination="destination address"}

45. MimeTEX Bot

  1. Replaces TeX math language enclosed in {tex} tags to beautifully-renderred formulas through the use of cgi binaries, so you do not need to have TeX Apache modules installed on your server.
    Get the cgi file here: http://www.forkosh.dreamhost.com/source_mimetex.html
    Syntax: { tex} math here{/tex}

46. MosDrop

  1. Display a dropped capital at the beginning of a paragraph.
    No tag needed.

47. Hide Intro Text

  1. This bot hides introtext when viewing complete items, but leaves the introtext intact on home page / navigation pages..

48. nabToggleDisplay

  1. Creates a configurable Show/Hide toggle on specified content. Content can be text, images, complex tables, whatever...
    Syntax: { nabToggle title="Some Title"}SomeContent{/nabToggle}

49. Smart

  1. ICQ Status

    Shows an online status indicator for any ICQ# in content items
    Syntax: { icq=123456789}
  2. OptimizeTables

    Table optimization runs daily (mambot checks file /media/optimizetables.flag last modification time).
    No tag needed.

50. Double Keywords

  1. Remove double keywords from the head-section. Joomla takes all keywords from the global configuration and from all content items that are displayed on the pages, eg. in a blog category. If there are any double keywords, this plugin will remove them.
    No tag needed, just publish.

51. botMosXslt

  1. Allows processing of an xml/xsl file using xslt processor, and renders the result in content items.

52. Registered User Read

  1. Hides enclosed text from non-registered visitors.
    Syntax: { mosregread }Your hidden information { /mosregread }

53. Xe-VidMambot FLV

  1. Displays Adobe Flash .flv Movies in your content. Progressive streaming of FLV Videos in your content, play and pause - sound ON/OFF buttons, Autoplay mode, Volume Control, Stretch Movies, Title description, Backgroundcolor settings, Popup supported, resize movie width and height...
    Syntax
    : { xemediabot}[1]|[2]|[3]|[4][5][6][7][8]{/xemediabot }
    1 - your media file, supported for flv, you can use absolute, relative and full URLs
    2 - false for inline, true(link title) for displaying a link with "link title" as linktext to popup
    3 - width of displayarea/popup
    4 - height of displayarea/popup
    5 - movie text description
    6 - autoplay on=1 / off=0
    7 - stretch movie on=1 / off=0
    8 - backgroundcolor - example: 0xAEB8BE(is like #AEB8BE)

54. Read All for registered users

  1. Select categories and/or sections where intro text will be availible for all users and the rest of your content is visible for registered users only.
    I am not linking this bot as the download requieres you to register on their site. :(

55. EasyBot

  1. Easy interface to load mambots.
    Syntax
    : Use function hsbot('') to load a single mambot, or hsbots('') to load an entire mambots group.

56. MosLaTeX

  1. Converts tagged LaTeX equations into PNG files on the fly. The images are then displayed inline in place of the original formula. Note that to use moslatex you require working latex, dvips and convert binaries.

57. Inline Links

  1. Creta links as menu items so you have all associated parameters available

58. TagBot

  1. Tagbot is a mambot/plugin that gives visitors to your site the ability to bookmark any article (content item) in one of the many Bookmark or Tagging sites such as Furl.net, Spurl.net, Del.icio.us etc.
    No tag needed, check parameters when you publish the bot


59. Google analytics plugin

  1. Inserts G.A. tracking code into the head tag of your template.
    No tag needed.

60. SM2Session

  1. Because of the way that Joomla! has been programmed the session table is only ever cleaned up when someone logs in to the backend. For a lot of sites this may never happen - creating some rather big session tables.
    Also, it only does a delete, whereas this plugin will do a delete, check and repair if necessary. This reduces the overhead also.
    The mambot runs from the frontend, ensuring that the session table is cleaned up more often. By default, it will randomly run 1 in a 100 requests...You can also force it to run everytime, or by using a link variable that will force it to run if given the right secretword.
    All configuration is done in the file rather than using params, this way we reduce the number of calls to the database.
    Installation is easy.. simply install as you would any other mambot.


61. AllVideos Bot

  1. Easily embed online streaming videos inside your content (items). No more HTML tags!
    Supported video providers are currently Google Video, YouTube, iFilm, Vimeo and Metacafe

62. RunPHP

  1. Execute PHP code inline from within content items

63. JSmarty

  1. Smarty template engine adapter. This plugin can be used for following template routines: set the configuration parameters; assign a value to a template parameter; display/fetch a template page.

64. Xe-mp3mambot

  1. Display MP3/SWF Files in content. Progressive streaming of MP3 files, play and stop buttons, Autoplay/Autostop mode, Volume control, Miniplayer with Description or Miniplayer Mini, Times display indicator, Backgroundcolor supported, popup supported, resize width and height supported.
    Commercial.

65. meat strip_tags

Strip out unwanted html formatting from content. Solves the problem with users pasting word formatted text in wysiwyg editors.

Using Pyzor with SpamAssassin

Source: http://atmail.com/view_article.php?num=285

Description
Pyzor is a collaborative, networked system to detect and block spam using identifying digests of messages. You can use this along with SpamAssassin to identify spam for incoming mail.

To use Pyzor with SpamAssassin, just run through the following steps:

- download the pyzor source from http://pyzor.sourceforge.net

% wget http://domain.com/pyzor-x.x.x.tar.bz2


- unzip, untar:

% tar xvfj pyzor-x.x.x.tar.bz2

- setup, and install:

% cd pyzor-x.x.x
% python setup.py build
% python setup.py install

- run pyzor 'discover' to download Pyzor server locations:

% pyzor discover

- afterwards, create a dummy text file called 'foo.txt'. test Pyzor by running the following:

% /usr/local/atmail/spamassassin/bin/spamassassin -D <>
- in the text dump that will follow, you should see the following:

debug: Pyzor is available: /usr/bin/pyzor

Pyzor will then be integrated into your installation.

Configuring a mail server with Postfix-Procmail-Fetchmail-SpamAssassin-ClamAV-Courier IMAP

Source: http://www.jennings.homelinux.net/mailserver_config.html

Configuring a mail server with Postfix-Procmail-Fetchmail-SpamAssassin-ClamAV-Courier IMAP



After struggling for days to get my mail server working as I wished I put
this piece together both as an 'aide memoire' to myself, and the
possible help of others.



This configuration is quite comprehensive. Many stages can be left out if you only require a simple mailserver setup.





Configuring a Postfix mail server.



Mail is received both directly via SMTP , and from ISPs mail servers using fetchmail.


On receipt mail is checked for Windows viruses using ClamAV, and then checked for
Spam using Spamassassin and will be placed in a Qmail style Mail directory.



A Qmail directory uses one file for each mail message which is moved between three different folders. By comparison
a standard Unix mail file is one long file with new mails appended to the end.
Maildirs are used in this application because the Courier-IMAP application
requires them. In turn Courier-IMAP is used in preference to another
IMAP/POP3 server because of its ability to disassociate mail boxes
from System logins for greater security.



Clients will access their mail in two ways




  • Local users use Kmail as a client to read mail either directly from the Maildirs, or using IMAP.

  • Remote users will use Kmail or Sylpheed to access the Maildirs using IMAP
    protocol over SSH. For additional security these users will be not
    be using their normal login password, but will use a different
    password. The Courier-imap authentication daemon will perform the
    mapping between the mail password, and the login password. Because
    IMAP is being used, all mails remain on the mail server so users can
    check their mails either locally or remotely and see exactly the
    same thing.



SoftwareUsed





  • Mandriva 2005LE Operating System

  • postfix-2.1.5-6mdk


  • fetchmail-6.2.5-10mdk

  • fetchmail-daemon-6.2.5-10mdk

  • procmail-3.22-5mdk

  • spamassassin-3.0.4-0.1.102mdk (

  • spamassassin-spamc-3.0.4-0.1.102mdk

  • spamassassin-spamd-3.0.4-0.1.102mdk

  • spamassassin-tools-3.0.4-0.1.102mdk

  • perl-Mail-SpamAssassin-3.0.4-0.1.102mdk

  • clamav-0.83-6mdk


  • clamav-db-0.83-6mdk

  • metamail-2.7-11mdk

  • lha-1.14i-11mdk (in contrib)

  • unarj-2.43-19plf (in plf)

  • unrar-3.30-2plf (in plf)

  • courier-imap--3.0.8-5mdk

  • courier-imap-pop-3.0.8-5mdk

  • TrashScan v0.12 from http://trashware.mirrorz.com/


  • In Mandriva 2006 the packages spamassassin-spamc and spamassassin-spamd have been left off the CDs. They are however in the 'main' folders on the online mirrors. Refer to http://easyurpmi.zarb.org/ to set up an online urpmi source for 'main'.








Make sure that the 'mailman' package is not installed.
If when installing Mandriva you select the 'Mail' tick box during package
selection, the installer will install mailman as well as Postfix.
Mailman alters the config files of Postfix, and Postfix will not work so long as
mailman is unconfigured. It is much easier to simply not install mailman.




Fetchmail Configuration


Mail comes from a variety of sources some of which are various ISP who
provide access via POP3 protocol. Fetchmail is used to read mail from
the ISP mailbox and pass it over to a local MTA (Mail Transfer Agent- Postfix) which
in turn will pass it to a Mail Delivery Agent (Procmail) for final delivery.




Fetchmail may be used either in single user mode, or in system wide mode.
In this case I am using fetchmail in system mode to collect mail for all
users.




The fetchmail-daemon RPM allows fetchmail to be started as a system
service which is configured by the file /etc/rc.d/init.d/fetchmail



This file will by default cause fetchmail to poll the ISPs POP3 servers every 180 seconds.


The configuration of fetchmail is controlled by the file
/etc/fetchmailrc which is easily configured with an editor.



My/etc/fetchmailrc now looks like this :-




set postmaster derek
set bouncemail
set no spambounce
set properties
poll pop.ispsrus.com with proto POP3
user 'username' there with password 'obscured' is 'derek' here options fetchall antispam 501
poll email.isp.net with proto POP3 interval 2
user 'myuser' there with password 'obscured'is 'derek' here options fetchall antispam 501
poll pop.anisp.com with proto POP3 interval 10
user 'anotheruser' there with password 'obscured' is 'rosie' here options fetchall antispam 501




Note: The 'antispam 501' option tells fetchmail that if Postfix refuses to
accept the mail with an error 501, that it is OK to discard the mail as spam and
not leave it on the POP3 server. I had to put this in because some Spam mails
had illegal headers which Postfix refused to accept.

Note: The 'interval N' option tells fetchmail to only poll that server once every 'N'
poll cycles. So for example N=2 will cause the server to be polled every 6 minutes.




By default fetchmail
will pass each mail as it is received to port 25 on the local host, or if
there is nothing listening on port 25, the mails will be passed to procmail. So there
is no point testing fetchmail until Postfix is installed and listening on Port
25.




PostfixConfiguration


Postfix is used as the mail server in favour over the most popular mail server on the internet Sendmail, because of its
much more simple configuration, and its reputation for security. It is also the mail server of choice
of MandrivaSoft, and I figured they probably know better than I do. There are many other mail server applications which could
be used in place of Postfix such as Qmail, Courier, and Exim.



Postfix may be configured either by editing text files, or by a Webmin
module. If you have never used webmin, then simply install the webmin module, and in Mandriva ControlCentre>System>Services start
the webmin service and then point your browser at

 https://localhost:10000

Note: https NOT http

You will then be able to manage many applications including Postfix with a web based GUI. In this guide however I am going to describe configuring by editing the configuration files directly since you will get a much better understanding of what is going on.




Configuring /etc/postfix/main.cf


In this application very few parameters needed changing from the system
defaults.




  • Postfix will complain if your computer does not have a hostname and domain name defined (Such as
    MyComputer.anyolddomain.co.uk, or myhost.localdomain) Simply calling your computer 'Fred' is not acceptable.The hostname may
    be set using drakconnect in Mandriva Control Centre, or by editing /etc/sysconfig/network .
    If you do not want to use your Fully Qualified Domain Name as the name of your computer, then it is essential to set the
    myhostname parameter in the Postfix configuration file.

Here are my changes to the /etc/postfix/main.cf file




command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
myhostname = mail.mymailaddy.net
mydestination = $myhostname, localhost.$mydomain, $mydomain
mynetworks = 192.168.1.0/24, 127.0.0.0/8
home_mailbox = Maildir/
mail_spool_directory = /var/spool/mail
alias_maps = hash:/etc/postfix/aliases
mailbox_command = /usr/bin/procmail -Y -a $DOMAIN
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
manpage_directory = /usr/share/man
debug_peer_level = 1
debugger_command = PATH=/usr/bin:/usr/X11R6/bin xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
delay_warning_time = 4
relayhost = smtp.myisp.com





The trailing '/' in the
home_mailbox line is very important. It defines Qmail type Mailboxes
instead of the default Unix type. This is important because I shall
be using the courier-imap IMAP server which requires Qmail format.



I am on an ADSL service from my ISP which does not guarantee I will always have the same IP address. Normally I would not be able to run a mailserver on such a service because people sending me mails need to translate between a domain name and an IP address. I get around this problem by using a Dynamic DNS Service.



Dyndns.org provides a free DNS service. You can choose your own Fully Quallified Domain Name, and whenever anyone tries to send you a mail their mail server will be directed to your IP address. The Dynamic DNS service is informed any time your IP address changes by a little daemon called ddclient available as an rpm package on any Mandriva contrib mirror. Ddclient runs as a service under Mandriva. Just edit the ddclient file in /etc to define your dyndns.org login and your hostname.



The relayhost line is needed because I am on a cable modem and when I send mails some recipients will perform a reverse DNS lookup to check if the IP address matches the hostname in the header. Since I use Dynamic DNS the reverse DNS will return my ISPs host name and my mails will be rejected. Forwarding mails via my ISPs SMTP server gets around this issue.




In
order to have the mail for 'root' to be forwarded to a user, an entry
is added to the /etc/postfix/aliases file. Be aware. Postfix will continuously put error messages in the log until a root alias has been defined.




#Person who should get root's mail
root:derek


As well as defining a root alias we shall also define two other aliases called 'ham' and 'spam'. Their use will become clear when we discuss Procmail and Spamassassin.





#Account to receive redirected spam and ham mails
spam:root
ham: root


Now create the aliases database by typing in a root terminal :-



newaliases
postfix reload



Virtual Addressing


In my application I am
receiving emails for two different domains, my normal domain, and a domain I
host for some friends.I therefore need to activate a virtual domain in Postfix.
This is done by adding a line to /etc/postfix/main.cf



virtual_maps = hash:/etc/postfix/virtual


The file /etc/postfix/virtual is then edited to include the lines



friendsdomain.homeip.net friendsdomain.homeip.net
postmaster@friendsdomain.homeip.net postmaster
sue@friendsdomain.homeip.net sue
grant@friendsdomain.homeip.net grant


An alias database must then be created by executing the command




postmap /etc/postfix/virtual

and then users must be created for the users on the Linux system. Since
these users do not require log on access to the Linux computer.They can be
created without a /home directory




adduser -c Grant -M -p xxxxxx grant

Then restart Postfix and you should be able to
receive mails addressed to the virtual domain.




ProcmailConfiguration



The mailbox_command parameter in the postfix configuration
causes procmail to run as the final delivery agent, and to use ~/.procmailrc as
its configuration file.




Procmail is organised as a sequence of 'recipes'. Each recipe starts with a test, followed by an action if the test is met.



In my example the first recipe tests the 'To: ' header of each mail. If the mail is 'To: spam@foobar' then this mail is a spam mail which had previously made it to my inbox, and I had redirected it in order that spamassassin could 'learn' it's characteristics and become better at detecting spam.



The second recipe is similar, but this time it is non-spam mail which had been falsely recognised as spam. (Note : I have never actually had spamassassin falsely detect a non spam mail as spam)
In both cases 'sa-learn' is invoked to train spamassassin.


Note : Training spamassassin with sa-learn will improve its effectiveness, but even without these recipes Spamassassin is very effective.


The third recipe calls trashscan which will run the mail through the ClamAV anti-virus engine. Infected mails will go into the 'virus' mail folder, and emails will be sent to alert the recipient, and the mail administrator.

Note : Virus protection is only required if your mail server is going to have clients using Windows. Effectively there are no Linux viruses so you will not need ClamAV if all your clients are Linux users.



The next recipes detect if the mail has come from a newsgroup to which I am subscribed, and will place the mail in a folder for that group.



The last recipe runs Spamassassin and depending on the result will put spam mails into a junkmail folder.
Any mail which does not meet any of the procmail recipes will go into the default folder.





[root@jennings root]# more /home/derek/.procmailrc

SHELL=/bin/sh
PATH="$HOME/bin:/usr/bin:/usr/local/bin:/usr/include:/usr/local/sbin:/bin:/sbin:/usr/sbin"
LOCKFILE=$HOME/lockfile.lock
ASSASSINLOCK=$HOME/assassin.lock
DEFAULT=$HOME/Maildir/
JUNKMAIL=$HOME/Maildir/.junkmail/
TRASH=$HOME/Maildir/.Trash
NEWBIE=$HOME/Maildir/.LinuxNewbie/
EXPERT=$HOME/Maildir/.LinuxExpert/
VIRUS=$HOME/Maildir/.virus/
BITBUCKET=/dev/null
LOCKTIMEOUT=10
#LOGFILE=/tmp/procmail_log
#LOGABSTRACT=no
#VERBOSE=no


# Feed redirected spam to sa-learn
# If a spam does make it past spamassassin I redirect it to spam@mymailaddy
#It will then be added to the spam database

:0
* ^To:.*spam@foobar
* < admin="postmaster@mymailaddy" quar="~/Maildir/virus" mta="/usr/sbin/postfix--">


#Filter out Mails from Newgroups
#There is no need to test these for spam

:0 :
* ^Sender: newbie-owner
$NEWBIE

:0 :
* ^Sender: expert-owner
$EXPERT


#Junk mail from unwanted sources
:0
* ^From: Sms Message
$BITBUCKET


#Run SpamAssassin daemon mode
#spamc is the daemon for spamassassin
#spam mails will come back marked with a spam header
:0 fw : $ASSASSINLOCK
| spamc


:0
* ^X-Spam-Flag: YES
$JUNKMAIL


# Catches everything else.
#Anything left over goes into the DEFAULT folder






SpamAssassinConfiguration



SpamAssassin needs little
setup other than to install the RPM, and then start the spamd service in Mandriva Control Centre>System>Services.




Spamassassin runs many tests on each mail. Some of them are textual (looks for spammish words like viagra), others test the mail headers (check for header spoofing), while others look for other spammer behaviour (such as BIG FONTS)



One of the most effective tests is Bayesian analysis.



Bayesian analysis works by comparing the words in a mail with words that appear in known spam, and known non spam, then calculating the probability of the mail being spam.
In order for Bayesian analysis to work it must have a database of known spam and non -spam (ham) to work on.

Spamassassin will not use Bayes until it has a database of 200 spam mails and 200 non spam mails.


Each time a mail has a score above 12 points, Spamassassin will add it to its spam database, and will add it to its ham database if the score is below 0.1 points ( I set this to 1.0 in my network)



If you have folders of spam or ham mails you can use them to initialise the Bayes database using the utility 'sa-learn'. See 'man sa-learn' for details.



Whenever a spam mail gets past spamassassin and makes it through to my inbox, I redirect it back to my mailserver addressed to spam@foobar where it goes back to my procmail recipes where it is recognised, and the mail is passed through sa-learn to improve spamassassin. (Replace foobar with the address of your mail server)






When spamassassin is first run it will create a file ~/.spamassassin/user_prefs which may be modified to
suit your taste.



By default spamassassin misses out some tests which
are free for personal use, but require a subscription from third
parties for commercial use. Since those tests are very useful it is
worth adding them.



You can also change the score threshold for
detecting spam, and add addresses to the whitelist if they keep
getting detected as false positives.




There is also a global configuration file at /etc/mail/spamassassin/local.cf which will set the defaults for all users.

Here is mine :-



# This is the right place to customize your installation of SpamAssassin.
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
###########################################################################
#
#rewrite_subject 0
#report_safe 1
auto_whitelist_path /var/spool/spamassassin/auto-whitelist
auto_whitelist_file_mode 0666
use_bayes 1
bayes_auto_learn 1
use_razor2 1

# How many hits before a mail is considered spam.
required_hits 6

# score SYMBOLIC_TEST_NAME n.nn
score RCVD_IN_BL_SPAMCOP_NET 4






The autowhitelist is a feature that will recognise if people have sent you spam or non spam (ham) mails in the past and weight theur scores accordingly. So it is unlikely spamassassin will reject mails from your regular correspondents.



Here is my /home/derek/.spamassassin/user.prefs file which will override the global file. :-




dns_available yes

# Bayes filters requires at least 200 entries of spam and 200 of ham
# for start working
use_bayes 1
bayes_auto_learn 1
bayes_auto_learn_threshold_nonspam 1
skip_rbl_checks 0
#If Vipul's razor is installed it can be used to check
#if a mail has been reported as spam
use_razor2 1
#pyzor is another database of reported spams
use_pyzor 0
#My spamasssin crashes if I enable dcc checking
use_dcc 0
dcc_timeout 5
#dcc_add_header 1



# How many hits before a mail is considered spam.
required_hits 5

# Whitelist and blacklist addresses are now file-glob-style patterns, so
# "friend@somewhere.com", "*@isp.com", or "*.domain.net" will all work.
# whitelist_from someone@somewhere.com
whitelist_from *.mandrakeclub.com

blacklist_from *@recessionspecials.com
blacklist_from *@amazingwebspecials.com


# Add your own customised scores for some tests below. The default scores are
# read from the installed spamassassin rules files, but you can override them
# here. To see the list of tests and their default scores, go to
# http://spamassassin.org/tests.html .
#
# score SYMBOLIC_TEST_NAME n.nn
# I change these because when Bayes is enabled other scores are
#too low in my opinion
score MISSING_OUTLOOK_NAME 0.100 0.576 0.1 0.1
score HTML_60_70 0.527 0.100 0.200 0.200
score HTML_SHOUTING3 0.108 0.333 0.1 0.1
score HTML_TAG_EXISTS_TBODY 0.496 0.201 0.439 0.200
score MISSING_MIMEOLE 0.501 0.501 0.241 0.200
score MISSING_OUTLOOK_NAME 0.100 0.576 0.1 0.1
score IN_REP_TO 0 0 0 0
score REFERENCES 0 0 0 0
score BAYES_60 0 0 2.502 2.502
score BAYES_70 0 0 3.637 3.637
score BAYES_80 0 0 4.1 4.1
score BAYES_90 0 0 4.2 4.2
score BAYES_99 0 0 4.300 4.3






It is possible to extend spamassassin to do additional tests.
I like to check a variety of DNSBL locations so I can check for example if a mail has come from China for example. I know no one in China, so to receive a mail from there is very suspicious. To enable DNSBL checking create a file /etc/mail/spamassassin/dnsbl.cf containing the text here


(Thanks to Bryan Phinney for providing this list)


It is possible to add many more customised rule sets. A visit to SpamAssassin Rules Emporium can provide
you with many custom rule sets, and a utility called "RulesDuJour" to automatically update them.



Clam Anti Virus Configuration




Once Clamav has been installed run 'freshclam' from a root terminal to bring the virus database up to date.




However the clamav RPM package will have put a cron job in /etc/cron.daily to update the virus database every day. Unfortunately the way this job is written you will get a warning email from cron even if the update completes successfully.


Therefore I have altered /etc/cron.daily/freshclam like this :-



#!/bin/sh

# A simple update script for the clamav virus database. This could as well
# be replaced by a SysV script.

# fix log file if needed
LOG_FILE="/var/log/clamav/freshclam.log"
if [ ! -f ${LOG_FILE} ]; then
touch $LOG_FILE
chmod 644 $LOG_FILE
chown clamav.clamav $LOG_FILE
fi

/usr/bin/freshclam \
--quiet \
--datadir=/var/lib/clamav \
--log=$LOG_FILE \
--log-verbose \
--daemon-notify=/etc/clamav.conf

es=$?
if [ $es="1" ]; then
exit 0
else
exit $es
fi



It should be remembered that cron will not run jobs which were missed because the computer was switched off, and since cron.daily runs at 4am each morning the jobs will never be run if you switch the computer off at night. To avoid this issue install the 'anacron' package which will run missed jobs when the computer is powered on.




A simple shell script called trashscan is used to pass mails through the ClamAV scanner.
Trashscan used to be included in the ClamAV package, but is no longer provided. You can download the latest version from
Here. (currently 0.12)
Unpack the compressed folder and put the trashscan file into /usr/local/bin/trashscan edit the variables to suit your environment and make the
file executable.Here are the changes I made to trashscan:-




SCANDIR=/tmp
VSCANNER=clamav
VSCANPRG=/usr/bin/clamscan
VSCANSUSP=virus
ALERT=yes
ALERTRCVR=postmaster@foobar # Receiver of virus alert messages
ALERTSNDR=postmaster@foobar # Sender of virus alert messages
ALERTCTCT=postmaster@foobar
NOTIFY=no
#(${CAT} ${TASKDIR}/head.rep; ${CAT} ${TASKDIR}/body.rep) | ${SENDMAIL} -t -oi # Comment this line out to suppress notification to sender
# they are all spoofed anyway, so there is no point

Replace 'foobar' with the name of your mail domain.



Installing metamail is required for trashcan, and installing unrar, lha, and unarj will enable trashscan to find viruses in compressed archives.



In operation Procmail will pass all mails through trashscan which will run clamscan on them. A header is placed in the mail indicating whether it is suspicious or not, and the next procmail recipe will put infected mails into a virus mail folder. A warning email will be sent to the mail recipient, and to the mail administrator.
Being a Bash script trashscan is not suitable for use on high volume mail servers. I am using it because I was having trouble with clamdmail which I was previously using.







Maildir Configuration


Our remote users will be serviced by Courier-Imap which only
supports Maildir type mailboxes, so our local users are also going to use Maildir
format for consistency. (This also allows local users to retrieve their mail when
travelling).



A Maildir mailbox
consists of a directory in the users $HOME with a default name of Maildir. Inside
this directory are three other directories with the names cur, new, and
tmp



Subfolders may also be placed in the Maildir
directory. Their names should be prefixed with . To denote them as a
subdirectory. The remote clients will be using Sylpheed which requires specific
names for certain folders.
The easiest way to define mail folders is using the maildirmake++ command
which is part of the courier-imap package.
In the users home directory enter this command (as the user not root)
:-




maildirmake++ Maildir
maildirmake++ -f Sent Maildir
maildirmake++ -f Queue Maildir
maildirmake++ -f junkmail Maildir
maildirmake++ -f virus Maildir
maildirmake++ -f Drafts Maildir
maildirmake++ -f Trash Maildir




YourMaildir will then have this structure









Now we can test out using Kmail to receive mails directly from the Maildir folder.
When first starting Kmail it will default to Unix style mailboxes (flatfiles) To make Kmail use Maildir
folders :-




Settings>ConfigureKMail>Folders
Select "By default mail folders are :-directories(maildirformat)"
Nextyou must define an account to recieve mails on
Settings>ConfigureKMail>Network>Receiving>Add
Select'Maildir mailbox'>OK
Pick a name for the account, select 'location'= ~/Maildir
Enable Interval mail checking.



Next we will define a sending
account

Settings>ConfigureKMail>Network>Sending>Add

SelectSMTP account>OK

Name=pickaname

Host=Your_host_name.your_domain_name>Port=25>OK


When you restart kmail it should find the Maildir folder which will be listed in addition to the
normal kmail folders.<>
Note: We are not yet using IMAP to access the mails. IMAP is actually more flexible than accessing the maildirs directly with kamail, so I recommend that IMAP should be used even when the mail server is local.






Testing Local delivery


If all is correct (and I have written this guide
correctly) you will now be able to test mails which originate from your local
computer back to yourself.



Using kmail send a mail to your local mail address (make
sure you use the smtp account we just set up)

your_user@localhost



The mail will be delivered from Kmail to postfix which will recognise this as a local user, and pass the mail to procmail, then to spamassassin, then to /home/your_user/Maildir/new where it will be found
by Kmail after a short wait who will then put it into~/home/Mail/inbox/cur


If your mail does not arrive trace it back through the system to see where it is stuck. Do not forget to look in /var/log/mail/errors,/var/log/mail/info, and the procmail log if
enabled.



The relayhost=smtp.myisp.com statement in /etc/postfix/main.cf is used to tell postfix which mail server to relay mails
through which terminate somewhere else on the internet. Postfix could send mails
directly to the final destination in a single hop without going through a relay
server, but many mail servers will compare the host names in the header of a mail
with the IP address listed in DNS servers, and reject them if they do not match on
the assumption the mail has been sent by a spammer trying to impersonate another
mailserver. By putting my ISPs mailserver name in this field the header of the
mail will have the same name and IP address as the previous hop, and receiving mail
servers will be satisfied.



If you send a mail using Kmail to your ISPs account you can check if
you can send mails using postfix, and that fetchmail is corrrectly configured to
receive them using POP3 protocol.



If the mails never arrive back in your inbox stop the fetchmail daemon and run fetchmail manually. In a root terminal :-



service fetchmail stop
fetchmail -v -f /etc/fetchmailrc
Examine output then
service fetchmail start






Configuring Courier-imap




The next step is to enable IMAP access to allow mail clients to log in to check their
mail.



Install the courier-imap RPM.


Courier-imap is started by the command



service courier-imap start


or by setting the service in Mandriva Control Centre>System>Services



If OpenSSL is also installed courier-imap will automatically support
logins over ssl when requested.



With default settings courier-imap will work with the configuration
described so far. By default courier-imap will use the directory 'Maildir' in a users home. If you
require it to use a different directory edit the 'MAILDIRPATH=' parameter in /etc/courier/imapd (and pop3d, pop3d-ssl) file.
There are also lots of other intereseting parameters in the courier-imap configuration to play with.



To use imap with sylpheed


In Sylpheed


Configuration>CreateNewAccount


Name=pickaname

Protocol=IMAP4

Server for receiving=your_mail_servername.and_domain

smtpserver

(send)=your_mail_servername.and_domain

UserID=your_user_name

Password+your_password

SelectSSL Tab and set

UseSSL for IMAP4 connections>OK


Right clicking on the account and selecting Rescan Folder Tree should cause an IMAP login, and you should see your mail and folders.



If you do not see all the folders you defined in ~/Maildir, then
that is probably because they do not start with a '.' character, or they are
not Maildir type directories with a cur,new, and tmp subdirectory. It is also
essential that maildir directories named .Sent , .Drafts, and .Trash are
present.



You should now be able to send and receive mails from Sylpheed.using IMAP.




Using remote Imap folders with Kmail



Create a new account with Settings>ConfigureKmail>Network>Recieving>Add>Imap.
Choose a name for the account and give the connection details. Once the account is added you will see
it appear in Kmail's folder list.



Right click on the base Imap folder and select 'Subscription'. A window will appear to allow you to select which of the subfolders
are going to be visible. Once a folder is subscribed to it will appear under the base folder. Right clicking on a subfolder
and selecting 'Properities' allows you to select if the subfolder is checked for new mail.



Now close Kmail, restart it, and go back to Settings>ConfigureKmail>Network>Receiving and Modify your Imap account.
You will now be able to select your Imap Trash folder as the 'Deleted Items folder'. Restarting Kmail is necessary because
the Imap folder will not appear in the drop down selection box if the base folder has only just been subscribed to.



Similarly go to Settings>ConfigureKmail>Identities, select an identity to modify>Advanced, and select your Imap Drafts
and Sent folder as 'Drafts folder' and 'Sent Mail folder'




Configuring Courier-imap-pop



Some of my users need POP3 access to their mails. The courier-imap-pop package supports pop3 and pop3 over SSL. It uses the same Qmail folders as the Imap server, but can only support one folder. So all that is
required is to create a single mail folder



maildirmake++ Maildir

The Pop3 service will not start until the file /etc/courier/pop3.dist is renamed to /etc/courier/pop3 (for normal POP3) and/or the file /etc/courier/pop3d-ssl.dist is renamed to /etc/courier/pop3d-ssl (for
secure pop3 access)



Then restart the courier imap service with



service courier-imap restart

And both imap and pop3 services should be running.





If you are not concerned with increasing system security the mail server configuration is now complete, and you can give yourself a pat on the back.



Securing your System


Using the same username/password combination for login and for remote mail access is a
security risk.



If your username/password were to be intercepted when retrieving e-mails, then
they could be used to attack your computer with telnet (if you were foolish
enough to enable it), or even ssh (if you did not depend on public/private key
combinations)




So I decided to enforce a system whereby my remote users had a different
password to the local login.




The simplest way of configuring this with courier-imap is to use 'authuserdb'
authenticationwhich is possible with the standard courier-imap RPM without any
additional modules.




Normally when you use imap authentication is performed by PAM. Any remote users
who offer a valid login password will be granted access.
To disable this; edit the file/etc/courier/authdaemonrc and comment out and
insert as shown




#authmodulelist="authcustom authcram authuserdb authldap authpgsql authmysqlauthpam"
authmodulelist="authuserdb"




This will disable all methods of user authentication other than the
'userdb' method.



Restart the authdaemond with the commands




/usr/lib/courier/authlib/authdaemond stop


/usr/lib/courier/authlib/authdaemond start



You will now find your imap access no longer works!!!



Now we must create a user database in the file /etc/userdb




First we have to pick passwords for each form of access we want to permit, e,g
imap, pop3, pop3 secure etc.



A password is created using the command



userdbpw

When prompted give the password you wish to use. An encrypted string will be
returned.



Now create the file /etc/userdb in the format shown below for
each local user who will require remote E-mail access. Where I have put
'obscured' insert the encrypted password string. You must insert a
password string for each method of access you want to enable, e.g.
impappw,pop3pw, pop3spw. Be sure also to insert the correct
uid/gid and home directory information for each user.
The uid/gid numbers may be ascertained by typing ' /usr/sbin/pw2userdb'




derek uid=501|gid=501|home=/home/derek|shell=/bin/bash|imappw=obscured|pop3pw=obscured




Note:There is one Tab character after the username.There must beNO
spaces



Now rebuild the binary database with the commands



/usr/lib/courier/authlib/authdaemond stop

makeuserdb

/usr/lib/courier/authlib/authdaemond start



Your imap server should resume working. The authentication daemon will be looking
up the password supplied by remote IMAP clients in the binary database for the rquested service type, and only
allowing access if there is a match.



Restart the authdaemond again, and change your password in your mail
client(e.g. sylpheed) to be the new password (unencrypted), and your Imap should
be working again.




Firewall Considerations -
To send/receive mail via SMTP port 25 must be open. To check mail with Pop3 from a computer in the local network, port 110
must be open. (Do not open port 110 to the Internet unless you want to be able to check mail remotely). To retrieve mail with IMAP
port 143 must be open. A convenient tool to manage the shorewall firewall is webmin. Install the webmin RPM, start webmin service using
Mandriva control Centre>System>Services, and then enter https://localhost:10000 in a browser.




Wednesday 11th August, 2004











Snap OS v3 with Active Directory

Source: http://wiki.procooling.com/index.php/Snap_OS_v3_with_Active_Directory

Snap OS v3 with Active Directory
From Pro/Wiki

Join the *NIX machine to your Active Directory (see WinBind docs)

Turn on NFS sharing in the Snap Server.

Edit the root$ share and add an NFS entry for UID 0 from the IP of your *NIX machine.

Make a directory on the *NIX machine for a NFS mount - e.g. mkdir /mnt/snap

As root, mount the root$ share of the snap using NFS from the *NIX machine

e.g. # mount -t nfs ip.of.snap.server:/root$ /mnt/snap

Edit your /etc/samba/smb.conf to share the mounted folder to Active Directory users.

This should get you up and running. Hope someone finds this useful.

Postfix SpamAssassin Procmail

Source: http://traxel.com/doc/spamassassin-setup.html

This is a very barebones setup for Postfix / SpamAssassin / Procmail. It is not performance optimized (in testing it handled about 25 mails per minute on an Athlon 900) look into spamc/spamd if you have higher volume.

This all assumes the following rig:
Debian Woody, fairly minimal installation.
Postfix installed and functional.
Pine and UW-IMAPD as the clients.

If your rig differs from the list above, so will your installation. This is just what worked for me.

Step 1:
This is important: You have to start with a fully functional install of Postfix. If it's not working right before you start, it will still not be working right when you're done, and it will be doing more things. That will make it a lot harder to fix.

Step 2:
Read all the way through this. It's not long, and it will prepare you for what's ahead.

Step 3:
Accept all responsibility for your actions. It is my deep and heartfelt belief that if you do what I've listed here it will destroy your computer, cause your refrigerator to defrost while you're on vacation, and make your girlfriend leave you. None of that happened to me, but if you choose to proceed, anything that happens is your responsibility.

Download the latest version of SpamAssassin
http://www.spamassassin.org/downloads.html
For most software, I use Debian's apt utilities to keep up to date. But with the rapid pace of development of spam, having the latest weapons is worthwhile.

Install Perl HTML::Parser
# apt-get install libhtml-parser-perl

Make SpamAssassin
$ tar -xvzf Mail-SpamAssassin-2.60.tar.gz
$ cd Mail-SpamAssassin-2.60
$ perl Makefile.PL
$ make
$ su
# make install

Create the filter user
Do this however you like. Filter should have a * password, tmp as a home directory, and a valid shell. As root, you will want to su filter for testing, but filter should not be able to log in.

Make sure procmail is installed
$ which procmail
/usr/bin/procmail

Configure SpamAssassin /etc/mail/spamassassin/local.cf
required_hits 8
add_header all Level _STARS(X)_
rewrite_subject 1
subject_tag Spam [_HITS_]

Create /usr/local/bin/sa-filter.sh
#!/bin/bash
/usr/local/bin/spamassassin | /usr/sbin/sendmail -i "$@"
exit $?

Set the access rights for sa-filter
# chown filter:filter /usr/local/bin/sa-filter
# chmod u+x /usr/local/bin/sa-filter
# chmod go-x /usr/local/bin/sa-filter

Test sa-filter
# su filter
$ cat spam.txt | sa-filter -f bob -- bob
$ cat notspam.txt | sa-filter -f bob -- bob

The previous test should leave you with both emails in your inbox, and the spam one should be marked spam. If not, don't move on until it works right. As things stand right now, postfix is still working exactly as it was before - no harm done.

That's it for su'ing to filter. Set filter's home directory to something nonexistent and shell to /bin/false now.

Abandon Hope All Ye Who Enter
Remember the part about your computer, refrigerator, and relationship all crashing? This is where it all happens. Remember, it's not my fault.

Make a backup copy of /etc/postfix/master.cf
# cp /etc/postfix/master.cf /etc/postfix/master.cf.preassassin

Make a backup copy of /etc/postfix/main.cf
# cp /etc/postfix/main.cf /etc/postfix/main.cf.preassassin

Edit /etc/postfix/master.cf:
Add the filter
# Spam Filter
spamassassin unix - n n - - pipe
user=filter argv=/usr/local/bin/sa-filter -f ${sender} -- ${recipient}
Change the settings for smtp and smtpd:
smtp inet n - n - - smtpd
-o content_filter=spamassassin:
smtp unix - - n - - smtp
-o content_filter=spamassassin:

Add Procmail to /etc/postfix/main.cf
mailbox_command = /usr/bin/procmail -f- -a "$USER"

Bounce Postfix
# /etc/init.d/postfix reload

Test Mailing (from a different machine):
$ cat spam.txt | /usr/sbin/sendmail your-user@your-host.com
$ cat notspam.txt | /usr/sbin/sendmail your-user@your-host.com

This test should do the same as before, you should get both mails in your inbox, and the spam one should be marked as spam.

If this is not what happened, restore the postfix configuration main.cf and master.cf.

Set up procmail

Create the .procmail directory
$ cd ~
$ mkdir .procmail

Create ~/.procmailrc
PROCMAILDIR=$HOME/.procmail
LOG=$PROCMAILDIR/pmlog
# VERBOSE=yes # turn this on for debugging
MAILDIR=$HOME/mail # this is for pine - yours may differ
INCLUDERC=$PROCMAILDIR/rc.spam
# INCLUDERC=$PROCMAILDIR/rc.morefilters
# If none of the filters match, it will go to your inbox.

Create ~/.procmail/rc.spam
:0:
* ^X-Spam-Level: XXXXXXXX
zSpam

Test Mailing (from a different machine):
$ cat spam.txt | /usr/sbin/sendmail your-user@your-host.com
$ cat notspam.txt | /usr/sbin/sendmail your-user@your-host.com

This time, the non-spam should show up in your inbox, and the spam should go to the folder zSpam.

If this is not what happened, restore the postfix configuration main.cf and master.cf.

And you're done.

Copyright 2003 by Robert Bushman.
Creative Commons License
This work is licensed under a Creative Commons License.