Versioning .NET builds with MsBuild Tasks using TFS Revision or Git Revision

MSBuild - Detecting latest TFS Revision on Project Folder level using tf.exe history and replaces the Revision in AssemblyVersion and AssemblyFileVersion for AssemblyInfo.cs prior to building the project with CustomTasks in csproj.

Inspired by

• http://stackoverflow.com/questions/25768653/versioning-net-builds/25806365#25806365
• https://intovsts.net/2015/08/24/global-net-versioning-strategy-assemblyinformationalversion/

I set off to create an MsBuild Tasks file that could be imported into csproj files and that calls tf.exe history as command line to get the latest revision on a project folder and embedding in the build assembly version revision number. I did not want this to be hidden in a build server step and I wanted the code to be open if a developer needed to investigate the tasks.







The end result is shared in this Gist:

(Updated with Git support on 2017-07-28)

Adding Visual Studio Code Map Scrollbar

Since Visual Studio 2013 a code map has been available instead of a plain scroll bar.

Toggling is done from "Scroll Bars" in the settings tab either under All languages or under a specific language. 

I prefer this setting, with the "Wide" Source overview.

Sync an FTP folder using WinSCP commandline

syncftp.cmd

"C:\Program Files (x86)\WinSCP\winscp" /script="C:\FTP-Sync\download.winscp"

download.winscp

open ftp://user:password@server.com/
synchronize local "C:\FTP-Sync\public" /
exit

Batching Commands for SQL and Optimizing network round trips.

Optimizing Database Access And Diving Into .Net SqlClient by Mladen Prajdić
http://www.sqlpass.org/24hours/2016/summitpreview/Sessions/Details.aspx?sid=53738

I was watching the session and found that I would like to add to the batch commands section. Back in 2003 i wrote a query optimization model that still works to this day that i would like to share.

Problem 

You have a lot of data you need to import, pushing it one row at a time is very inefficient, but what is the "right" amount of queries to push.

Solution

SQL commands are sent through a network packet to the server, these packets are default 4096 bytes / 4KB in size, this can be defined on the system but most leave this setting intact. This means that if you write an insert for one row and sends that to the server there are plenty of unused space in the packet of 4096. 

So the task at hand is very simple, you must optimize your query so that you send as many queries in one packet of 4096 but without going over, since it then sends the overflowing query in another packet with trailing empty space as a result. 

If you watch the session by Mladen Prajdić there is a reference to a System.Data.SqlClient.SqlCommandSet that is used internally by the .Net framework, my guess is that it works similarly and tries to optimize the network traffic to the server. 

A more brutal implementation (clean and simple) is the write a batch query engine yourself and measuring the byte length of the resulting query and then only pushing it to the server when the byte length is close to 4096 but not over. 

I have shared my base code for this brutal approach on https://gist.github.com/janhebnes/726811a5ba0d36c9de7c323ae177bf22

Excel Losing Decimal Values When Value Pasted from SSMS ResultSet

http://blog.sqlauthority.com/2012/09/24/sql-server-excel-losing-decimal-values-when-value-pasted-from-ssms-resultset/

Remember to format cells to text in Excel, before pasting from the SQL Server Management results window.

This will keep the decimal format from SSMS as a dot and not autoformat the number and loosing the comma in the process.

Making Umbraco or any .Net CMS respect default documents

HttpHandler that helps respect the Request flow and serves the default documents of a directory instead of allowing e.g. Umbraco to kidnap the request for there purposes.

IIS does not serve the Default Document when set in Integrated pipeline mode. E.g. after installing Umbraco the httpModules will catch any traffic to the site and will return a 404 page instead of actually passing to the existing file system. In this case I wanted to add umbraco on top of an old asp and html based site. We did not want to move large amount of archive content, so using this modules allows for the classic Default Documents to be respected even if you are in a modern setup where Default Documents are being ignored. So first we check the file system for a response document if non is found the request is allowed to continue to Umbraco, allowing us to extend the existing content with Umbraco instead of having to nuke everything.

visual studio web deployment / deploying extra files

Deploying extra files with Web Deploy 

http://www.asp.net/mvc/overview/deployment/visual-studio-web-deployment/deploying-extra-files

The official search result shows a somewhat complex approach to including files to the project deployment, a more simpler approach is to fake that the files have been included as content. This approach requires the files to be accessible at the root prior to deploying.

By creating a *.wpp.targets (named the same as your project) you can add to the project files source prior to building. If you had included your content from Visual Studio to the project, it would have resulted in the external content being saved as <ItemGroup> and Content Include xxx for each file. By doing this manually you can use search patterns to add unknown amounts of files using \**\*.

ProjectName.wpp.targets

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <ItemGroup>
  <Content Include="css\**\*" />
  <Content Include="content\**\*" />
 </ItemGroup>
</Project>

It is that simple to include external content.

Another classic issue using webdeployment is to control the content of your web.config, AutoParameterizationWebConfigConnectionStrings is the solution for this. http://blog.jan.hebnes.dk/2015/07/umbraco-to-azure-msdeploy-error.html

Localization vs. Internationalization (i18n) in SQL and .NET

Internationalization is often written i18n, where 18 is the number of letters between i and n in the English word. Internationalization typically entails: Designing and developing in a way that removes barriers to localization or international deployment.

More about the term is described elegantly at w3
http://www.w3.org/International/questions/qa-i18n

----

There are many issues in handling time in systems that span globally.
One area is the database level and the other is the code level.

Prior to the introduction of the DateTimeOffset type the types offered did not let you save both the UTC time and the actual local timezone in one value. This would typically be handled by composing time upon two stored values, the datetime in UTC and the TimeZoneOffset in another field.

These shortcomings have been handled with the introduction of the DateTimeOffset value in both SQL and .NET. There is still room for improvement because we do not have the Daylight Saving Time Parameter in the type, but this is a value that is set nationally and just a pain to handle ;)

What have the larger systems been doing prior to this?

E.g.The Sitecore platform has saved all internal datetime values to the database as UTC time and let the presentation layer handle the formatting to localized time. For internal values like a publishing pipeline or an event queue having a single point of time reference is critical. This is still a relevant for system values where you want to be precise in the time comparison.

DateTimeOffset (from SQL server 2008)

http://blogs.msdn.com/b/bartd/archive/2009/03/31/the-death-of-datetime.aspx

SQL Server 2008 added a new data type named “datetimeoffset”. This is similar to the old datetime data type, with the following significant differences:

• Internally, the time is stored in unambiguous UTC format
• The local time zone offset is stored along with the UTC time, which allows the time to be displayed as a local time value (or converted to any another time zone offset)
• The data type is capable of storing more precise times than datetime

DateTimeOffset (from .NET Framework 3.5)

The same type has been introduced in .NET v3.5, it can map the new SQL type.
https://msdn.microsoft.com/en-us/library/system.datetimeoffset%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396

Choosing Between DateTime, DateTimeOffset, TimeSpan, and TimeZoneInfo
https://msdn.microsoft.com/en-us/library/bb384267%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396

The Daylight Saving Time - the missing link... 

I have found one excellent resource for handling Daylight Saving Time. The idea is to let the user select there country of origin, and save the Daylight Offset value at the same time, it can also save the gmt offset if you choose.

The right way is to find the value through the .NET framework with the TimeZone information for a region. But allowing it to be part of a custom value set by the user selection can help give a simplicity around the value. 

http://www.freeformatter.com/time-zone-list-html-select.html

C# Sending secure e-mails that are signed and encrypted

Securing email content can be done by sending the email in a closed network setup, directly to the receiving email server or by encrypting the content of the email. If you require the content to be really confidential you should go all the way and authenticate the sender by signing the email with a private certificate from the sender and encrypt the same email with the public certificate of the receiver.

This opens up a couple of failure points since certificates expire.
The following Gist handles both the signing and the encrypting but also takes care of notifying when the certificates have or are going to expire, and handles when they are expired.

Pressing ctrl+key in a WPF Window Application

We had a bug on a WPF application where KeyUp="MainControl_KeyUp" had been used for detecting ctrl+B

if (e.Key == Key.B && (Keyboard.Modifiers & ModifierKeys.Control) == ModifierKeys.Control)

This meant that only the B + right control would trigger the event.
Changing the event to being a KeyDown event and this post help solve the issue so both control keys now can be used to detect the key combination in the event context.

private void TextBox_KeyDown(object sender, KeyEventArgs e)
        {
            if ((e.Key == Key.G) &&
                (Keyboard.IsKeyDown(Key.LeftCtrl) || Keyboard.IsKeyDown(Key.RightCtrl)))
                MessageBox.Show("You pressed Ctrl+G !");
        }

http://wpf.2000things.com/2012/08/17/627-detecting-whether-the-ctrl-key-is-pressed-in-a-keydown-event-handler/
http://wpf.2000things.com/tag/keydown/

Turn on Compile-time View Checking for ASP.NET MVC

The .cshtml files are not natively compiled and therefore cheched for spelling mistakes at compile time. This can be changed, e.g. at build server level.

<target condition="'$(MvcBuildViews)'=='true'" name="AfterBuild">
<aspnetcompiler physicalpath="$(ProjectDir)\..\$(ProjectName)" virtualpath="temp"></aspnetcompiler>
</target>

<propertygroup>
<mvcbuildviews>true</mvcbuildviews>
</propertygroup>

This has been an option since 2010, not many know about it.
http://blogs.msdn.com/b/jimlamb/archive/2010/04/20/turn-on-compile-time-view-checking-for-asp-net-mvc-projects-in-tfs-build-2010.aspx

D3js visualization of Steam Group relations between friends and games using Steam API

Last year at the lanparty http://netbyte.dk I visualized the Steam Group relations between friends and games using a D3js sample and Steam API.

http://netbyte.dk/steam/games/
http://netbyte.dk/steam/

It was done as a code and run feature for generating the source file that could be used for the above renderings.
Attached is the source for generating the flare files.

Sitecore Code Generation based on T4

Sitecore Code Generation based on T4 and extending the sitecore base template fields with class information for mapping the c# code to correct types.

AutoGenerated.tt is used by Visual Studio to generate the .cs file with all the template classes.
T4XmlProvider is the service that generates the Xml that is required by the AutoGenerated.tt to function.
Fields are mapped to types with information in the configuration and information stored directly on the Sitecore template field.

This model was developed back in 2010 and now shared with everyone, we no longer use the model but it can serve as an inspiration to others.

C# Operator ^= boolean flag toggle

Reviewing some code I spotted an interesting ^= operator that I had not seen before.

isBackgroundWindowOpen ^= true; // Toggles boolean flag.

C# Operators reference:
https://msdn.microsoft.com/en-us/library/6a71f45d.aspx

x ^= y – XOR assignment. XOR the value of y with the value of x, store the result in x, and return the new value.

Set Opacity in Powershell and cmd Console Windows

With windows 10 we finally have support for opacity in the console, so we can look cool when shelling... the opacity option can be adjusted in the default options.

It came in the first editions of windows 10, but I first found out now, so other might need a nudge to find out too. Several improvements have been implemented: https://blogs.windows.com/buildingapps/2014/10/07/console-improvements-in-the-windows-10-technical-preview/

It also kicks in on the PowerShell

Visual Studio Map mode for vertical scroll bar

This feature allready came in VS2013... 

umbraco set date field value from code

Had problems with umbraco not setting date field value from code, when you reloaded the interface.

My conversion code was:

var content = contentService.GetById(3150);
//content.SetValue("publicationDate", dateValue);
content.Properties["publicationDate"].Value = dateValue;
contentService.SaveAndPublishWithStatus(content, raiseEvents: true);

Notice both formats for entering values is usable. 

The resulting data was that the value was being written to the database in the date field

SELECT TOP 1000 *

  FROM [Umbraco].[dbo].[cmsPropertyData] where contentNodeId like '3150'

When dealing with a date field that has a prior value you can see the changed value right away when refreshing the browser, when working with an empty date you have to recycle the apppool for the date to come up in the interface. 

Umbraco to Azure / Transformed Web.config MSDeploy error

Could not open Source file: Could not find a part of the path

A rather simple umbraco nuget based install and base Publishing profile from Azure getting me these types of errors:

Transformed Web.config using D:\Solutions\xxx\src\xxx.Web\Web.Release.config into obj\Release\TransformWebConfig\transformed\Web.config.
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.targets(2294,5): Error : Could not open Source file: Could not find a part of the path 'D:\Solutions\xxx\src\xxx.Web\umbraco\Install\Views\Web.config;\umbraco\Install\Views\Web.config'.

Others having the same issue.
http://forums.iis.net/t/1217423.aspx?MSDeploy+error+Could+not+open+Source+file+Could+not+find+a+part+of+the+path+

Issue

Our default projects have a .wpp.Targets file that handles packages for msdeploy on our central build server, the package template for umbraco contains all files related to umbraco leaving "no one behind" strategy. 

wpp.Targets defines default packages information that is set throughout any publishing profiles.

Combined with the default publishing profile where a connection string wants to replace content in web.config this gives issues on web.config files other places than the root of the site. 

Our wpp.Targets contains a simple content inclusion, that makes sure the msdeploy package contains all required umbraco files even tho they are not part of the VS Project.

<ItemGroup>
  <!-- Umbraco -->
  <Content Include="css\**\*" />
  <Content Include="content\**\*" />
  <Content Include="macroscripts\**\*" />
  <Content Include="masterpages\**\*" />
  <Content Include="properties\**\*" />
  <Content Include="umbraco\**\*" />
  <Content Include="umbraco_client\**\*" />
    
  <Content Include="App_Browsers\**\*" />
  <Content Include="App_Plugins\**\*" />
  <Content Include="App_Start\**\*" />
 </ItemGroup>

http://www.asp.net/mvc/overview/deployment/visual-studio-web-deployment/deploying-to-production
A .pubxml file contains the settings that pertain to a specific publish profile. If you want to configure settings that apply to all profiles, you can create a .wpp.targets file. The build process imports these files into the .csproj or .vbproj project file, so most settings that you can configure in the project file can be configured in these files. For more information about .pubxml files and .wpp.targets files, seeHow to: Edit Deployment Settings in Publish Profile (.pubxml) Files and the .wpp.targets File in Visual Studio Web Projects.

Solution: AutoParameterizationWebConfigConnectionStrings

Several places online the proposed solution was AutoParameterizationWebConfigConnectionStrings=false as a parameter on msbuild but using VS Publishinging it was not clear how to introduce this parameter.

It turns out that you just through it in the "PropertyGroup" of the publishing profile. 
<AutoParameterizationWebConfigConnectionStrings>False</AutoParameterizationWebConfigConnectionStrings>

This is ninja kungfu knowledge that is apparently only learned after several hours of fidling about... any msbuild property can be set on the "PropertyGroup" of a profile.. enjoy.

The only change left is that the release profile must replace the connection string to the production connection string is otherwise would be set on the publishing replace event.

Double bonus solution

The PropertyGroup> <AutoParameterizationWebConfigConnectionStrings can be set in the Wpp.Targets file and thereby allowing for a "default" wpp.Targets that can handle imported profiles from Azure without requiring changes in the imported profile pubxml.

Our wpp.Targets on all Umbraco projects is now:

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <ItemGroup>
  <!-- Umbraco -->
  <Content Include="css\**\*" />
  <Content Include="content\**\*" />
  <Content Include="macroscripts\**\*" />
  <Content Include="masterpages\**\*" />
  <Content Include="properties\**\*" />
  <Content Include="umbraco\**\*" />
  <Content Include="umbraco_client\**\*" />
    
  <Content Include="App_Browsers\**\*" />
  <Content Include="App_Plugins\**\*" />
  <Content Include="App_Start\**\*" />

 </ItemGroup>
 <PropertyGroup>
  <AutoParameterizationWebConfigConnectionStrings>False</AutoParameterizationWebConfigConnectionStrings>
  <AfterAddIisSettingAndFileContentsToSourceManifest Condition="'$(AfterAddIisSettingAndFileContentsToSourceManifest)'==''">
     $(AfterAddIisSettingAndFileContentsToSourceManifest);
     MakeEmptyFolders;
  </AfterAddIisSettingAndFileContentsToSourceManifest>
 </PropertyGroup>
 <Target Name="MakeEmptyFolders">
  <Message Text="Adding empty folders" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\App_Data" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\App_Code" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\css" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\imagecache" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\macroscripts" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\masterpages" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\media" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\usercontrols" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\views\partials" />
  <MakeDir Directories="$(_MSDeployDirPath_FullPath)\views\macropartials" />
 </Target>
</Project>

node install bypass Visual Studio 2010 build tools

node install error "please install Visual Studio 2010 build tools" 

Running node install on a recent project that i had not build before got me the error message pasted below, related to msbuild 2010 in a node context.
At first i believed there was an error in path because of build.js in "Failed at the node-sass@1.0.3 install script 'node build.js'" and maybe the path to msbuild got cought up.
But after an extensive search i finally came upon this nice little comment:


https://github.com/TooTallNate/node-gyp/issues/154#issuecomment-30200420

I installed VS2012, and now have to include --msvs_version=2012 in any npm install calls that will include native modules. (seanmonstar)

Reading the full error message one could find this statement:

The build tools for Visual Studio 2010 (Platform Toolset = 'v100') cannot be found. To build using the v100 build tools, please install Visual Studio 2010 build tools.

The Solution: npm install --msvs_version=2013

Full error message

> node-sass@1.0.3 install D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass
> node build.js

child_process: customFds option is deprecated, use stdio instead.

D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modu
les\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (rebuild)
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.Cpp.Platform.targets(64,5): error MSB8020: The build tools for Visual Studio 2010 (Platform To
olset = 'v100') cannot be found. To build using the v100 build tools, please install Visual Studio 2010 build tools.  Alternatively, you may upgrade to the cur
rent Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Upgrade Solution...". [D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass\build\binding.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\12.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:269:23)
gyp ERR! stack     at ChildProcess.emit (events.js:110:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1074:12)
gyp ERR! System Windows_NT 6.3.9600
gyp ERR! command "node" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass
gyp ERR! node -v v0.12.4
gyp ERR! node-gyp -v v1.0.3
gyp ERR! not ok
Build failed
npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
npm ERR! node v0.12.4
npm ERR! npm  v2.10.1
npm ERR! code ELIFECYCLE

npm ERR! node-sass@1.0.3 install: `node build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-sass@1.0.3 install script 'node build.js'.
npm ERR! This is most likely a problem with the node-sass package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node build.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-sass
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     D:\Solutions\xxx\npm-debug.log

The Solution: npm install --msvs_version=2013

 D:\Solutions\xxx\xxx>npm install --msvs_version=2013
npm WARN package.json xxx@0.0.1 No repository field.
npm WARN package.json xxx@0.0.1 No README data
npm WARN package.json xxx@0.0.1 No license field.
/
> node-sass@1.0.3 install D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass
> node build.js

child_process: customFds option is deprecated, use stdio instead.

D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modu
les\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (rebuild)
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
cl : Command line warning D9025: overriding '/GR-' with '/GR' [D:\Solutions\xxx\node_modules\grunt-sass\node_modules\node-sass\build\bindi
ng.vcxproj]
......

[And it went on - Happy `til the end]

Sitecore Rejected-By-UrlScan /sitecore/shell/Controls/Lib/jQuery/jQueryUI/1.9.2

One of our dev servers began having problems loading 2 specific files in the Sitecore backend, they returned 404 error pages instead.

~/sitecore/shell/Controls/Lib/jQuery/jQueryUI/1.9.2/smoothness/jquery-ui-1.9.2.min.css
~/sitecore/shell/Controls/Lib/jQuery/jQueryUI/1.9.2/jquery-ui-1.10.3.min.js

Changing the folder name to 1 instead of 1.9.2 made the content appear, so it was not file security related. In the IIS logs we could see Rejected-By-UrlScan

2015-06-30 08:43:17 192.168.2.21 GET /Rejected-By-UrlScan ~/sitecore/shell/Controls/Lib/jQuery/jQueryUI/1.9.2/smoothness/jquery-ui-1.9.2.min.css 80 - 192.168.1.109 Mozilla/5.0+(Windows+NT+6.3;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/43.0.2357.130+Safari/537.36 404 0 0 41352

On the 16. june 2015 an update had been released to Microsoft URL Scan 3.1
The config file for URL Scan is in a ini file in a subfolder of the IIS server. The setting AllowDotInPath seems to be the cause of the problem on the server.

• AllowDotInPath=0By default, this option is set to 0. If this option is set to 0, URLScan rejects any request that contains multiple periods (.). This prevents attempts to disguise requests for dangerous file name extensions by putting a safe file name extension in the path information or query string portion of the URL. For example, if this option is set to 1, URLScan might permit a request for http://servername/BadFile.exe/SafeFile.htm because it thinks that it is a request for an HTML page, when it is actually a request for an executable (.exe) file with the name of an HTML page in the PATH_INFO area. When this option is set 0, URLScan may also deny requests for directories that contain periods.

Source: https://support.microsoft.com/en-us/kb/326444

This is defintively the issue.

But we found that changing the setting AllowDotInPath and restarting the IIS Service did not solve the issue. So we have been forced to uninstall URL Scan, and now need to revisit this issue later on to see if we can get this up again.