Rob Tiffany

Microsoft SQL Server Compact 4.0 is the next generation of embedded database from the SQL Server family, and it’s coming to an IIS/ASP.NET web server in your neighborhood.  I wanted to share the news first announced by Scott Guthrie and Ambrish Mishra on their respective blogs.

SQL Server Compact 4.0 will provide an easy and simple to use database storage for starter websites, and the features of Compact that wooed the mobile devices and desktop developers will now be available to the ASP.NET web application developers. In addition, the features and enhancements in SQL Server Compact 4.0 like the new Transact-SQL syntax for OFFSET & FETCH, higher reliability, seamless migration of schema and data to the higher SKUs of SQL Server, support for code-first & server generated keys in ADO.NET Entity Framework 4.0, and the ability to use Visual Studio 2010 to develop ASP.NET web applications for Compact 4.0 etc. will also excite the existing development community.

The first Community Technology Preview (CTP1) release of SQL Server Compact 4.0 is integrated with the WebMatrix Beta, which is a free ASP.NET web application development tool for Windows, and provides development & database support, and can easily publish/deploy websites to the web hosting providers. SQL Server Compact 4.0 CTP1 plays an important role in WebMatrix Beta, and enables WebMatrix Beta to be a small download, with a simple development experience, and contributes to the seamless end-to-end experiences from development through to publish. Scott Guthrie’s (ScottGu) blog has detailed information about the WebMatrix Beta and its different components. WebMatrix Beta can be installed in one-click through the Microsoft Web Platform Installer (Web PI) 3 Beta. In addition to the Web PI 3 Beta, SQL Server Compact 4.0 CTP1 is also available for download from the Microsoft download center.

1.    Specialty of SQL Server Compact:

SQL Server Compact has many specialties that are well liked by the developer community. The specialties of Compact, among others, are as listed below:

• Small Size: Easy to download with a small size of 2 to 2.5 MB and installs fast with no configuration needed.
• Program with .NET Framework APIs: Simple to use because of the familiar .NET Framework APIs, including ADO.NET and ADO.NET Entity Framework, and similar programming patterns.
• Supports Transact-SQL Syntax: Supports a rich subset of the Transact-SQL syntax supported by higher SKUs of SQL Server.
• Private (xcopy) Deployable: Xcopy deployable within the application folders and no admin privileges are needed. The database file contains no code and is safe to deploy along with the database. 
• In-process Database: Loads in the application’s process memory space and requires no inter-process communication.
• Integrated with Visual Studio: Applications can be rapidly developed using the integrated development experience in Visual Studio.
• Easy Migration to SQL Server: Smooth on-ramp to SQL Server when the applications requirements grow and there is a need to handle large amounts of data and high concurrent loads due to multiple users.
• Free Product with Free Tools: Compact is a free product, and is integrated with the free Express versions of Visual Studio and WebMatrix Beta, and requires low investment to start building and deploying applications.
• Simple Solutions for Syncing data with SQL Server: Schema and data can be synced easily with SQL Server using inbuilt and programmable technologies like Sync Framework (Sync FX) and merge replication.

 

2.    New Features in SQL Server Compact 4.0 CTP1

The first CTP release of SQL Server Compact 4.0 has a lot of features that will be useful to all of the development community of Compact, including the latest entrants, the ASP.NET developers. The different features of SQL Server Compact 4.0 CTP1 categorized separately are as described below:

2.1  Fundamentals

The set of features in this category are improvements that made Compact better, more reliable and secure. More details about the features are given below:

1. Higher Reliability – SQL Server Compact 4.0 CTP1 has been tested and certified to work with ASP.NET web applications and can handle the load of starter websites. Compact 4.0 CTP1 performs without any exceptions or access violations in scenarios where there are a number of concurrent connections and multiple requests are being made to the database.
2. Encryption Algorithm Update – Compact 4.0 CTP1 supports the SHA 2 encryption algorithms that provide a higher level of security for the encrypted Compact database files and are also needed for the FIPS compliance. The algorithms that were supported in the earlier versions of Compact are only supported in Compact 4.0 CTP1 for opening the database file to upgrade them to Compact 4.0 CTP1. Due to this, when Compact 4.0 CTP1 opens a Compact 3.5 database file, the Compact 3.5 database file will need to be upgraded to Compact 4.0 CTP1 using the System.Data.SqlServerCe.SqlCeEngine.Upgrade() API.
3. Setup Enhancements – SQL Server Compact 4.0 CTP1 setup and installation has been enhanced to ensure that the x86 and x64 components of SQL Server Compact install properly on a 64-bit machine. This is done to ensure that Compact 4.0 CTP1 supports both the WOW64 mode and the 64-bit native applications on the 64-bit machine without any problems, in both the conditions i.e. when Compact is deployed centrally using the exe installer or when it is privately deployed within the application folders. �
   1. In Compact 4.0 CTP1, the x86 exe will install on a 32-bit machine and the x64 MSI will install on a 64-bit machine only. The x64 MSI will install both the 32-bit and 64-bit Compact components on the 64-bit machine and due to this the x86 MSI is no longer needed to be installed on the 64-bit machine.
   2. If one wants to do a private deployment of Compact 4.0 CTP1, all the assemblies (both 32-bit and 64-bit) for the private deployment will be available in the %Program Files%\Microsoft SQL Server Compact Edition\v4.0\Private and in the %Program Files (x86)%\Microsoft SQL Server Compact Edition\v4.0\Private folder.

 

2.2  Improvements for ASP.NET Web Applications

1. Support for ASP.NET – Compact 4.0 CTP1 supports ASP.NET without the need for any specific configuration. The flag SQLServerCompactEditionUnderWebHosting is not required in Compact 4.0 CTP1 and has been removed.
2. Virtual Memory Reduction – The virtual memory that Compact uses has been reduced in Compact 4.0 CTP1. The visible difference is that if an application tries to open 40 to 50 simultaneous connections in SQL Server Compact 3.5 SP2, an ‘out of virtual memory’ exception will be hit. With Compact 4.0 CTP1, the application can easily open as many connections as needed to the database file up to the full extent of 256 connections without running out of virtual memory.
3. Medium Trust – Compact 4.0 CTP1 assemblies have been attributed with Allow Partially Trusted Caller’s Attribute (APTCA) and Compact 4.0 CTP1 can run in medium trust.

 

2.3  Develop Compact 4.0 CTP1 Applications with WebMatrix Beta and Visual Studio 2010

1. Seamless Migration of Data and Schema from Compact 4.0 CTP1 into SQL Server – The WebMatrix Beta can be used to script data and schema from Compact 4.0 CTP1 into the higher versions of SQL Server. The WebMatrix Beta can be also used to script data and schema from Compact 4.0 CTP1 to SQL Azure. The script to SQL Azure is in an early (alpha) stage for this first CTP release.
2. Designer Support in WebMatrix Beta – The WebMatrix Beta can be used to develop ASP.NET web applications with Compact 4.0 CTP1 and to manage the Compact database including running T-SQL queries against Compact.
3.  Designer Support in Visual Studio 2010 – An update to Visual Studio 2010 is also being worked upon that will provide support for using Compact 4.0 in Visual Studio 2010. Compact 4.0 will be also supported in Visual Studio 2010 Web Developer Express.

 

2.4  New Transact-SQL Syntax

1. Support for Paging Queries – Compact 4.0 CTP1 supports the T-SQL syntax for OFFSET & FETCH, and due to this paging queries can be run against the database file. A few sample T-SQL queries for OFFSET & FETCH that can be run against the sample Northwind.sdf database file are given below:

• The query below skips the first 10 rows and fetches all the other rows in the Employees table ordered by the Hire Date:

      SELECT * FROM Employees ORDER BY [Hire Date] OFFSET 10 ROWS;

• The first query below skips the first 10 rows and fetches the next 10 rows in the Customers table and the rows are ordered by the Customer ID. The second query skips the first 2 rows and fetches the next one row in the Orders table ordered by the Shipped Date. This query a variation of the first query:

      SELECT * FROM Customers ORDER BY [Customer ID] OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY;

      SELECT * FROM Orders ORDER BY [Shipped Date] OFFSET 2 ROWS FETCH NEXT 1 ROW ONLY;

2.5  API Enhancements

1. Seamless Integration with ADO.NET Entity Framework 4.0 (.NET FX 4) – Compact 4.0 CTP1 will support the code-first programming model of ADO.NET Entity Framework. In addition, the columns that have server generated keys like identity, rowguid etc. will be also supported in Compact 4.0 when used with ADO.NET Entity Framework 4.0 (this is the version of ADO.NET Entity Framework that released with .NET FX 4). The ADO.NET Entity Framework is working on a separate release and that will provide the code-first and server generated keys support for Compact.
   1. The Compact 4.0 CTP1 release only works with ADO.NET Entity Framework 4.0 that released with .NET FX 4. Using Compact 4.0 CTP1 with ADO.NET Entity Framework 3.5 SP1 that released with .NET Framework 3.5 SP1 will result in an error.

1. New API for GetSchema: Compact has a new API – System.Data.SqlServerCe.SqlCeConnection.GetSchema() – that can be used to get the schema from a Compact database file. The API is also used to provide the support for System.Data.Common.DbConnection.GetSchema. In the example below the GetSchema API is used to get the names of all the tables, columns, indexes, indexed columns and foreign keys from the sample Northwind.sdf database.

 

SqlCeConnection conn = new SqlCeConnection("DataSource=Northwind.sdf");

       try

            {

                conn.Open();

                DataTable dt = conn.GetSchema();

                dt = conn.GetSchema("Tables");

                dt = conn.GetSchema("Columns");

                dt = conn.GetSchema("Indexes");

                dt = conn.GetSchema("IndexColumns");

                dt = conn.GetSchema("ForeignKeys");

            }

            catch (Exception e)

            {

                //Fail

            }

1. New API for SqlCeConnectionStringBuilder: Compact is has new API – System.Data.SqlServerCe.SqlCeConnectionStringBuilder() – that let developers programmatically create correct connection string for Compact 4.0, and to parse & rebuild existing connection strings. The API is also used to provide the support for System.Data.Common.DbConnectionStringBuilder. In the example below the SqlCeConnectionStringBuilder performs checks for valid key/value pairs and creates a valid SQL Server Compact connection string.

 

       System.Data.SqlServerCe.SqlCeConnectionStringBuilder builder =

            new System.Data.SqlServerCe.SqlCeConnectionStringBuilder();

       builder["Data Source"] = "Northwind.sdf";

       builder["Mode”] = "Exclusive";

       Console.WriteLine(builder.ConnectionString);

The result is the following connection string:

Data Source=Northwind.sdf;Mode=Exclusive

3.    Known Issues in SQL Server Compact 4.0 CTP1:

The known issues in this CTP release of Compact 4.0 are listed below:

1. Compact 4.0 CTP1 needs Visual C++ 2008 Runtime Libraries (x86, IA64 and x64), Service Pack 1: The native DLLs of SQL Server Compact 4.0 CTP1 need the Microsoft Visual C++ 2008 Runtime Libraries (x86, IA64 and x64), Service Pack 1. Installing the SQL Server Compact 4.0 CTP1 using the Windows Installer (.exe) file, also installs the Visual C++ 2008 Runtime Libraries SP1. If SQL Server Compact 4.0 CTP1 is deployed privately in the application’s folder the following have to be present on the machine for SQL Server Compact to function properly:
   1. Installing the .NET Framework 3.5 SP1 also installs the Visual C++ 2008 Runtime Libraries SP1.
   2. Visual C++ 2008 Runtime Libraries SP1 can be downloaded and installed from the location given below: http://go.microsoft.com/fwlink/?LinkId=194827

Note that installing .NET Framework 2.0 or 3.0 or 4 does not install the Visual C++ 2008 Runtime Libraries SP1.

1. Uninstallation and repair of Compact 4.0 CTP1 from the command line does not work: Uninstallation of Compact runtime using command line options does not work in this CTP release. For uninstallation of SQL Server Compact runtime go to Add/remove programs (Start->Run->appwiz.cpl), and right click on the entry ‘Microsoft SQL Server Compact 4.0 ENU CTP1’ or ‘Microsoft SQL Server Compact 4.0 x64 ENU CTP1’ , and select uninstall from the menu.
2. Compact 4.0 development support in Visual Studio 2010: An update to the Visual Studio 2010 is being worked upon that will add the support for Compact 4.0. The update will also provide for the development support for Compact 4.0 in the Visual Studio 2010 Web Developer Express.
3. Code first programming model support for Compact 4.0 in ADO.NET Entity Framework: The Microsoft ADO.NET Entity Framework Feature Community Technology Preview 3 is an early preview of the code-first programming model for the ADO.NET Entity Framework 4.0. The code-first feature in the ADO.NET Entity Framework CTP3 release does not work properly with Compact 4.0. Future releases of code-first programming model for ADO.NET Entity Framework will provide support for code-first programming model for Compact 4.0.

 

Now go have fun with my favorite database!

-Rob