mike-obrien.net Resume Blog Labs
Tuesday, December 26, 2006

Ran into a weird problem with the following code. When I atempted to save the bitmap I recieved a "Parameter is not valid" error. I snagged this code from a VB.NET app I wrote and translated it, so I knew that it worked.

Bitmap ChartImage = GeneratePieChart(...);

EncoderParameter QualityParam = new EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 100);
EncoderParameters EncoderParams = new EncoderParameters();
EncoderParams.Param[0] = QualityParam;

MemoryStream ChartStream = new MemoryStream();

ChartImage.Save(ChartStream, GetEncoderInfo("image/jpeg"), EncoderParams);

Rob Gruen ran into the same issue and fortunately blogged about it here. Long story short the quality in the EncoderParamater must explicitly passed as a long:

EncoderParameter QualityParam = new EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 100L);

.NET | C# | Graphics
Tuesday, December 26, 2006 7:40:39 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [1]  | 
Thursday, December 21, 2006

We had a situation come up where we needed to quickly export about 3000 tables from SQL Server to CSV. We bought a tool in hopes that we could do this quickly (Why wont DTS export to multiple text files?!?), well the tool stunk and it would have literally taken a week to complete. So I wrote a two SP's that use BCP to export an entire database or a single table to CSV (Or use whatever delimiter and qualifier you want). I whiped them up pretty quick so they could probably use some fine tuning if you are going to use them in production. Two things to note:

1) In SQL Server 2005 you have to explicitly turn on the ability to use xp_cmdshell in order to make the BCP call.

EXEC master.dbo.sp_configure 'show advanced options', 1

RECONFIGURE

EXEC master.dbo.sp_configure 'xp_cmdshell', 1

RECONFIGURE

2) Since the table selection is built on the fly (To add delimiters, qualifiers, column names and escape chars in data that match the qualifier) tables with a lot of columns will fail. The selection statement is peiced together in a VARCHAR(8000) so there is limited space.

3) I'm creating a temporary view in which to pull the exported data from because the call to xp_cmdshell is limited to 1k in SQL Server 2000 (I tried it without the view in 2005 and it doesent appear to have this limitation). 

Here is the proc that exports the entire DB:

CREATE PROCEDURE [dbo].[ExportDatabase]
@Database varchar(256),
@OutputPath varchar(1000),
@OutputExtension varchar(50) = 'csv',
@Delimiter varchar(50) = ',',
@Qualifier varchar(50) = '"',
@IncludeColumnHeaders bit = 1
AS
BEGIN

DECLARE @TableName varchar(256)

DECLARE @ColumnCursorSQL varchar(500)
SET @ColumnCursorSQL = 'DECLARE TableCursor CURSOR FAST_FORWARD FOR SELECT name FROM ' + @Database + '..' + 'sysobjects WHERE type=''U'''
EXEC (@ColumnCursorSQL)

OPEN TableCursor

FETCH NEXT FROM TableCursor INTO @TableName

WHILE (@@FETCH_STATUS <> -1)
BEGIN

    EXEC ExportTable
    @Database,
    @TableName,
    @OutputPath,
    @OutputExtension,
    @Delimiter,
    @Qualifier,
    @IncludeColumnHeaders

    FETCH NEXT FROM TableCursor INTO @TableName

END

CLOSE TableCursor
DEALLOCATE TableCursor

END

Here is the proc that exports a single table:

CREATE PROCEDURE [dbo].[ExportTable]
@Database varchar(256),
@Table varchar(256),
@OutputPath varchar(1000),
@OutputExtension varchar(50) = 'csv',
@Delimiter varchar(50) = ',',
@Qualifier varchar(50) = '"',
@IncludeColumnHeaders bit = 1
AS
BEGIN

DECLARE @ColumnNameSQL varchar(8000)
DECLARE @SelectSQL varchar(8000)
DECLARE @ColumnName varchar(256)
DECLARE @FirstColumn bit

SET @FirstColumn = 1

SET @SelectSQL = 'SELECT '
SET @ColumnNameSQL = 'SELECT '

DECLARE @ColumnCursorSQL varchar(500)
SET @ColumnCursorSQL = 'DECLARE ColumnCursor CURSOR FAST_FORWARD FOR SELECT name FROM ' + @Database + '..' + 'syscolumns WHERE id=object_id(''' + @Database + '..' + @Table + ''')'
EXEC (@ColumnCursorSQL)

OPEN ColumnCursor

FETCH NEXT FROM ColumnCursor INTO @ColumnName

WHILE (@@FETCH_STATUS <> -1)
BEGIN

    IF @FirstColumn = 1
        SET @FirstColumn = 0
    ELSE
        BEGIN
            SET @SelectSQL = @SelectSQL + @Delimiter
            IF @IncludeColumnHeaders = 1
                SET @ColumnNameSQL = @ColumnNameSQL + @Delimiter
        END

    IF @IncludeColumnHeaders = 1
        SET @ColumnNameSQL = @ColumnNameSQL + '''' +
        @Qualifier + @ColumnName + @Qualifier + ''' AS ' + @ColumnName

    SET @SelectSQL = @SelectSQL + '''' + @Qualifier +
        ''' + REPLACE(CONVERT(varchar(8000), ' + @ColumnName +
        '), ''' + @Qualifier + ''', ''' + @Qualifier + @Qualifier + ''') + ''' + @Qualifier + ''' AS ' + @ColumnName

    FETCH NEXT FROM ColumnCursor INTO @ColumnName

END

CLOSE ColumnCursor
DEALLOCATE ColumnCursor

SET @SelectSQL = @SelectSQL + ' FROM ' + @Database + '..' + @Table

IF @IncludeColumnHeaders = 1
    SET @SelectSQL = @ColumnNameSQL + ' UNION ALL ' + @SelectSQL

DECLARE @ExportTempViewName varchar(50)
SET @ExportTempViewName = + 'ExportTemp' + REPLACE(CONVERT(varchar(36), NEWID()), '-', '')
DECLARE @ExportTempViewCreate varchar(8000)
SET @ExportTempViewCreate = 'CREATE VIEW ' + @ExportTempViewName + ' AS ' + @SelectSQL

EXEC (@ExportTempViewCreate)

SET @SQL = 'bcp "SELECT * FROM ' + db_name() + '..' + @ExportTempViewName + '" queryout "' +
    @OutputPath + '\' + @Database + '.' + @Table + '.' +
    @OutputExtension + '" -c -t, -T -S' + @@servername

EXEC master..xp_cmdshell @SQL

DECLARE @ExportTempViewDrop varchar(8000)
SET @ExportTempViewDrop = 'DROP VIEW ' + @ExportTempViewName

EXEC (@ExportTempViewDrop)

END

Thursday, December 21, 2006 10:56:11 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 

The XSL below illustrates how to alternate color using the mod operator and setting the style class. This allows to to create a general ledger look.

    <xsl:template match="log/entry">
        <div>
            <xsl:if test="position() mod 2 = 1">
                <xsl:attribute name="class">LogEntryHighlight</xsl:attribute>
            </xsl:if>
            <xsl:if test="position() mod 2 = 0">
                <xsl:attribute name="class">LogEntry</xsl:attribute>
            </xsl:if>
            <xsl:value-of select="."/>
        </div>    
    </xsl:template> 

Thursday, December 21, 2006 9:11:59 AM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Monday, December 18, 2006

This should actually be "System.Runtime.CompilerServices.IndexerName" from what I can tell. It appears that there is a typo in the MSDN docs (And elsewhere on the web) unless I'm just totally missing something here... Supposedly its in the mscorlib.dll but I couldnt seem to find a "CSharp" class anywhere under System.Runtime.CompilerServices either in v1.1 or v2.0. And other CSharp classes elsewhere did not contain the "IndexerName" attribute. Weird. The truth is out there...

.NET | C#
Monday, December 18, 2006 11:00:22 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Sunday, December 17, 2006

I needed an efficient and performant way to keep track of the current number of files/folders in a folder. Directory.GetFiles() was out of the question for a number of reasons. So I ended up using a two step process. First grab the current number of files/folders in the folder using the FindFirstFile() and FindNextFile() Win32 API calls (Which is what Directory.GetFiles() uses under the covers.). These increment the file counter. Then immediately after that completes, activate a FileSystemWatcher to monitor file creation and deletion. These events respectively increment or decrement the counter. I have been putting this through the ringer and it seems to do the job superbly.

Interesting observation about the FindNextFile() method; this method will continue to grab files as long as they exist in the directory, it’s not like it’s working from some sort of static snapshot of the files created by FindFirstFile() when it is called. So in other words if other files are added to the folder that match the criteria of your filter while you are in the process of iterating with FindNextFile(), FindNextFile() will eventually “find” them. Although this seems implied by the name of both the method calls, you never know what might actually be going on under the hood…

The following code illustrates how to do this. Note that there can potentially be a significant performance hit when a folder is first enumerated. This is especially true when the file/folder count gets past the hundreds of thousands. But once the folder is first enumerated the FSW will keep track of the additional creates/deletes, no need to re-enumerate.

class Program
{

   static FileSystemWatcher _Watcher = new FileSystemWatcher();
   static int _Count = 0;

   static void Main(string[] args)
   {

      _Watcher.Filter = "*.*";
      _Watcher.Path = @"D:\Temp";

      _Watcher.Created += new FileSystemEventHandler(OnCreated);
      _Watcher.Deleted += new FileSystemEventHandler(OnDeleted);

      NativeMethods.FindData FileInfo = new NativeMethods.FindData();

      IntPtr FileHandle = NativeMethods.FindFirstFile(@"D:\Temp\*.*", FileInfo);

      do
      {
         if (FileInfo.fileName != "." && FileInfo.fileName != "..") { _Count++; }
      } while (NativeMethods.FindNextFile(FileHandle, FileInfo) == true);

      NativeMethods.FindClose(FileHandle);
      _Watcher.EnableRaisingEvents = true;

      do
      {
         Console.WriteLine("Total Files: " + _Count.ToString());
      } while (Console.ReadLine() != "q");

   }

   private static void OnCreated(object source, FileSystemEventArgs e)
   {
      _Count++;
   }

   private static void OnDeleted(object source, FileSystemEventArgs e)
   {
      _Count--;
   }

}

class NativeMethods
{
   // Declares a class member for structure element.
   [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
   public class FindData
   {
      public int fileAttributes = 0;
      // creationTime was an embedded FILETIME structure.
      public int creationTime_lowDateTime = 0;
      public int creationTime_highDateTime = 0;
      // lastAccessTime was an embedded FILETIME structure.
      public int lastAccessTime_lowDateTime = 0;
      public int lastAccessTime_highDateTime = 0;
      // lastWriteTime was an embedded FILETIME structure.
      public int lastWriteTime_lowDateTime = 0;
      public int lastWriteTime_highDateTime = 0;
      public int nFileSizeHigh = 0;
      public int nFileSizeLow = 0;
      public int dwReserved0 = 0;
      public int dwReserved1 = 0;
      [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
      public String fileName = null;
      [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 14)]
      public String alternateFileName = null;
   }

   [DllImport("Kernel32.dll", CharSet = CharSet.Auto)]
   public static extern IntPtr FindFirstFile(String fileName, [In, Out] 
   FindData findFileData);

   [DllImport("Kernel32.dll", CharSet = CharSet.Auto)]
   public static extern bool FindNextFile(IntPtr hFindFile, [In, Out] 
   FindData findFileData);

   [DllImport("kernel32.dll", CharSet = CharSet.Auto)]
   public static extern bool FindClose(IntPtr hndFindFile);

}

.NET | C#
Sunday, December 17, 2006 8:45:09 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Monday, December 04, 2006

The following steps outline how to install and configure Subversion v1.4.

1. Download the latest subversion package from here (Described as "Windows installer with the basic win32 binaries").

2. Using svnadmin (In the Subversion bin folder) create a new repository: svnadmin create drive:\my\repository\path

3. Setup the Subversion server as a windows service as specified here. If you want to store repositories on drives other than the drive the subversion service is running on or you would like access to be restricted to a certain path you will need to specify a repository root. This can be specified in the service setup.

4. Configure security as specified here.

5. Install a Subversion client such as TortuousSVN and test access to the repository: svn://MyServer/MyRepo

Monday, December 04, 2006 10:51:42 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 

I want to start off by saying that I am running subserve v1.4 as a Windows service. From what I have read configuration may be somewhat different when using httpd, SSH or on another OS. So the following may or may not fully apply to other access methods, versions or OS’s.

svnserve.conf

The first repository security configuration file you will encounter is the svnserve.conf file. It appears that Subversion looks for this file specifically under the “conf” folder within the repository tree. I don’t believe there is a way to configure this path or the filename it looks for. This file has only one section called “general”. It contains 5 basic security settings. The settings are as follows:

anon-access: Valid values for this setting are read, write or none. This value specifies repository wide access for anonymous users. If there is a conflict with this setting and with folder permissions specified in the authz file, the most restrictive permission is applied.

auth-access: Valid values for this setting are read, write or none. This value specifies repository wide access for authenticated users. If there is a conflict with this setting and with folder permissions specified in the authz file, the most restrictive permission is applied.

password-db: This specifies a path to a user database. This can be an absolute path (“C:\config\passwd”) or a relative path (“..\..\config\passwd”). If no path is specified only anonymous access will be supported. Only the anon-access permission will be in force.

authz-db: This specifies a path to a folder permissions database. This can be an absolute path (“C:\config\authz”) or a relative path (“..\..\config\authz”). If no path is specified then path based permissions are not applied. Only the anon-access and auth-access permissions will be in force.

realm: This is a string that serves as a unique identifier which identifies a realm or security domain the repository is associated with. A realm is basically the user base contained in the user database (passwd file). Each repository that shares a single user database should have the same realm specified. So for example if you had a user database at “C:\config\passwd” which was used by three repositories, all three repositories should have the same realm specified.

passwd

The second file you will encounter is the user database. The name of this file is passwd by default but can be anything. If user authentication is desired the path to this file must be set in the password-db setting in the svnserve.conf file. If no path is set for this setting, no user authentication will take place. The user database can be shared by multiple repositories. Each repository that shares a single user database should have the same realm specified. The user database contains only one section called “users”. The settings in this group are username/password pairs. For example:

[users]
bsimpson =
R@dio@ctiveM@n
lsimpson = Bl33dingGumsMurphy
mburns=Smith3rs
djquimby=V0t3Quimby

authz

The third file you will encounter is the path based permissions database. The name of this file is authz by default but can be anything. If path based permissions are desired the path to this file must be set in the authz-db setting in the svnserve.conf file. This file defines user groups and path permissions. The first section in this file is called “groups”. Here you can specify groups and their membership. Each setting is a group name and a comma separated user list. It does not appear that white space in this list causes any problems. Unfortunately as of  Subversion 1.4 you cannot specify user groups in the user database (passwd), only in this file. Here is an example of the “groups” section:

[groups]
admin = mburns, djquimby
users =  bsimpson, lsimpson

The sections that follow are path specific permissions. The section name is the path. The path begins with a forward slash but must not end with one. Evidently (As of Subversion 1.4) the “repository” prefix for repository specific permissions (IE: [repository:/yada/yada]) is ignored when using svnserve to access repositories. It appears that it is only recognized when using the httpd access method. So separate files would have to be maintained to provide repository specific folder permissions. Otherwise if identical paths existed in two or more repositories the permissions for those paths would be identical, there would be no way to differentiate them. Each path permission section contains members which could be a group name prefixed with an ampersand (@), wildcard (*) denoting all users (Including anonymous) or simply a username. The permissions are ‘r’ for readonly, ‘rw’ for read/write or ‘’ (Blank) for no access (Including anonymous users). Permissions are inherited from the parent folder if no permissions are specified for a particular path. If permissions conflict, the least restrictive permissions are applied. Here is an example of path permissions:

[/common]
* = rw

[/config/admin]
@admin=rw
lsimpson=r

[/docs]
@users=rw
@admin=rw

[/secure]
@admin=r
@users=

Monday, December 04, 2006 10:08:48 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 

I am running svnserve as a windows service on our build server. I thought it was pretty cool that you could point repositories to central security configuration files. This is really nice if you have multiple repositories, only two files to maintain for each relm. Problem is though that when you set a repository specific folder permission ([repository:/yada]) in the authz file it is totally ignored. For example:

[repository:/reporoot/repo1]
@group1=rw

[repository:/reporoot/repo2]
@group2=rw

This does nothing! I did a little searching on "the google" and evidently the repository specific prefix only works with httpd access. For svnserve access you have to specify an authz file for each repository to specify repository specific permissions. Posts here and here discuss this issue.

Monday, December 04, 2006 7:51:12 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 

Many times when we think of OSS we think "cost savings" or “free”... Not so. There are many hidden costs that accompany the integration of OSS into your infrastructure. I’m currently in the process of integrating some OSS into our infrastructure and am really seeing this. Now don’t get me wrong I’m a big fan (and user) of OSS but it’s important to see the big picture when making decisions and planning projects that involve OSS.

One of the hidden costs is learning and configuring. OSS, from what I have seen, seems to be seriously lacking when it comes to ease of use and configurability. Many times OSS is command line based, configuration is in INI style config files. It takes time to read through documentation (Which may be sparse in some cases). I have also found that actually playing around with the app and configuration is in some cases the best way to figure out how to configure and use the software. Again, all of this takes time. On the other hand the pretty app that you pay for will (If it’s any good) have an intelligent installer, very intuitive GUI, and lots of support and documentation. I recently had an experience where I setup one of those “pretty, paid for” apps and I literally had it up and running and configured in a couple of hours. I was totally blown away! If it were OSS it would more than likely taken me a couple of days (Or more) to get in there and figure it out.

Another hidden cost I have seen is maintenance… Many OSS software packages have cryptic or crud management and/or maintenance interfaces. These can be cumbersome and time consuming to work with and can be difficult to get others up to speed on using them. This may also lead to neglect because it’s a pain to work with or others don’t feel they have the time to learn it. The app that you pay for (If it’s any good) will probably have a slick interface to maintain and administer.

Again, I’m not knocking OSS, I think OS development is a wonderful thing. There are a lot of awesome OS projects out there. I also think that OSS fills an important roll in any infrastructure. But when you decide to go the OSS route you can’t be in the mindset that because it’s OS it’s free. You really have to think about the pros and cons and make a decision based on that. In some cases it may actually be cheaper (And more advantageous) to buy instead of going the OSS route.

As a side note, I have heard some balk at “pretty, paid for” apps for various reasons, some valid some not objective at all. But the reality is this: Those days as a single guy staying up all night eating Doritos, drinking Mt Dew and hacking on all kinds of interesting stuff are over. In the real world you have a family, deadlines and clients (both internal and external). Time is really of the essence and you dont have much of it. Your boss or your wife and kids aren’t going to be impressed because you worked OT for 2 weeks learning how to wiz around some cryptic OSS like Kevin Mitnick phreaking a telco switch. The time spent getting to that point may not be worth it or you may not even have it. In many cases “pretty, paid for” equate to steep learning curve, ease of use and maintenance and good support. They also may mean less stress and more time for life’s more important things. It really comes down to making a good business decision that will help your business to succeed, not a decision based on ego that makes you think your The Brain.

Monday, December 04, 2006 5:38:37 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Thursday, November 30, 2006

You cant do it... Didnt realize this since I wouldnt normally do that (for performace reasons). But I was attaching an MDF that happened to be in my temp folder, which is compressed, and got the error. I didnt quite get it at first since I didnt realize this was not possible in 2005 and because I forgot that folder was compressed... :P You can however do this if the file is in a readonly file group or if you are attaching a readonly database.

The file "D:\temp\test.mdf" is compressed but does not reside in a read-only database or filegroup. The file must be decompressed.
CREATE DATABASE failed. Some file names listed could not be created. Check related errors. (Microsoft SQL Server, Error: 5118)

Thursday, November 30, 2006 5:00:45 AM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Wednesday, November 22, 2006

So I'm implementing an interface, defined in an assembly written in VB.NET, on a class in an assembly written in C#. The interface defines a number of events. VB.NET event declaration doesent require you to define the corresponding delegate (as C# does) so the question is where does the VB.NET compiler declare it for you after compilation? Well it seems that the delegate definition is declared right in the interface. This could be confusing for someone who doesent know VB.NET since C# does not allow the declaring of types within an interface whereas VB.NET does.

Here IDataInputDevice is an interface residing in an assembly written in VB.NET with three events defined. The corresponding delegate definitions are defined right within the interface.

.NET | C# | VB.NET
Wednesday, November 22, 2006 11:39:52 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  | 
Monday, November 20, 2006

I was looking over a code sample for reading from/writing to a serial port here at the MS support site. The VB.NET sample creates a SerialPort object within a using statement but then explicitly closes the SerialPort which is kind of an oxymoron. After doing a little investigation I found that the SerialPort class inherits from the Component class which implements IDisposable. The Dispose method in the Component class calls the Dispose(bool disposing) method which is overriden by the SerialPort class. The overriden method closes the SerialPort. In fact the Close method just turns around and calls this overriden method. Long story short, SerialPort.Dispose closes the serial port, so no need to explicitly close it within a using statement.

Reflected Component.Dispose method:

public void Dispose()
{
    this.Dispose(true);
    GC.SuppressFinalize(this);
}

Reflected SerialPort.Dispose method:

protected override void Dispose(bool disposing)
{
    if (disposing && this.IsOpen)
    {
        this.internalSerialStream.Flush();
        this.internalSerialStream.Close();
        this.internalSerialStream = null;
    }
    base.Dispose(disposing);
}

Monday, November 20, 2006 11:02:38 PM (GMT Standard Time, UTC+00:00)  #   |  Comments [0]  |