domingo, 21 de junio de 2009

Yahoo Developer Network – All APIs page

Today while was trying to make a custom control to convert currencies, I faced an ugly problem: The free soap web service that I was planning to use was simply down. A big problem considering the time I had to made that control.

After some goggling I found some sites that perform this kind of job. Some of them were using originally the same web service I was trying to use and because they were facing the same problem as me they decide to change it. They were using somehow the YAHOO finance quotes service to retrieve the exchange rate between two currencies, but they were making the calls directly at the server side (trying to do it using Ajax raises the Cross Site Scripting problem).

Then I was curious about to know if somehow the YAHOO finance service could had some kind of REST API that could help me to obtain this value directly. But I was not lucky and couldn’t find that…

But I found something even better the Yahoo Developer Network - All APIs page and there I found a lot of interesting things… really a lot. But the most impressive think I found was the YQL.

The YQL stands for Yahoo! Query Language. This is a really impressive concept. It basically allows to make calls to almost any web resource (html pages, dynamic pages and any other resource) and query this result using css or xpath to retrieve the parts of the content that are relevant for us in JSON or XML formats. And it supports make Cross Site Scripting calls too!

The potential that I found in this tool is really amazing you can retrieve almost any relevant information from many sites and use it in the way you want.

YAHOO has some good resources. Really good resources!

sábado, 20 de junio de 2009

Testing Windows 7

I just finished the installation of all the programs that I usually use every day on my computer with the RC1 of the Windows 7. And I must admit it was relatively easy to get it working fine. I had some troubles but they were quickly solved.

Some of the aspects about this new Windows version are really nice.

  1. The installation itself was easy. I remember I tried to install Vista RC1 a long time ago, It was just terrible, a lot of components of my computer just won’t work ever on that because the drivers for them simply didn’t exist. This time instead almost of my hardware components were ready to use, just after the installation. The Wireless USB receiver, the USB web cam and my two audio cards were the exception, but I didn’t have any trouble trying to install the vista version of the driver that came with the USB receiver, or with the driver version of Vista for my two audio Cards (Soundblaster Audigy 2 Platinum, and the default integrated audio that came with my Intel DP35DP Motherboard). I found the Driver version for the Intel Integrated Audio on the Intel website, and even when they claim that the driver was beta also it is doing really good right now.
  2. The Aspect of the Desktop. I don’t know if Microsoft copy the new desktop style from others Operative Systems, I will only say that it looks good, I seems a lot like Vista, but with less invasive warning windows (yes they’re still there). Well I must admit I have not used Vista too much, I really didn’t use Vista on a regular basis like to make a fair comparison. But comparing Windows 7 with the well loved Windows XP you really see some good changes in the UI. I’m tempted to believe that this instance of Windows 7 is running faster than my old good XP.

I will post more comments about Windows 7 later… for today it is time to stop!

Bye.

image

image

image

image

image

image

viernes, 19 de junio de 2009

Export to CSV in Unicode format so Excel can read Japanese characters in the right way

To make this history short, here is the problem I had. The Client wants to export the data from a report (basically a table) in csv format. This data could contain Unicode characters (basically Japanese characters). This file was intended to be used to move the data to other programs like excel or others.

Well to solve that requirement i made this method, that basically receive the data ready to be put in the csv format and save it to the response.

protected void ExportToCSV(object sender, EventArgs e)
 {
    if (String.IsNullOrEmpty(CSVData.Value)) return;

    var strFileNameexport = FileNameHidden.Value.Replace(" ", "_");
    Response.Clear();
    Response.ClearContent();
    Response.Buffer = true;
    Response.ContentType = "text/comma-separated-values";

    Response.ContentEncoding = Encoding.UTF8;
    Response.Charset = Encoding.UTF8.HeaderName;
    var data = CSVData.Value;    

    Response.AddHeader("Content-Disposition", string.Format("attachment;Filename={0}.csv", strFileNameexport));

    Response.Write(data);
    Response.End();
}

As you can see as far to this point everything seemed to be working fine. But suddenly when we try to open a exported file that contains Japanese characters, Excel was not able to display those characters. It replace those characters with garbage

So to Solve the Problem we...

  • found that excel expects the csv file to be in "utf-16le" (UCS-2 Little Endian) (weird, really weird).
  • when you put the data in this format, the "," character is not longer a valid column separator (even more weird, weird!!!!) so you need to use other character like "\t" (I know, I know... if you use \t as a separator the mime type should tab separated value files, but Excel seems to don't complain about it.
  • It is also recomendable that you enclose the data for each cell using quotes, this way the values inside won't be interpreted wrong if a separator character is present there.
  • then realize that we were receiving the data in utf-8 format so we first need to convert the bytes from utf-8 to utf-16
  • And the most important note: YOU NEED TO PUT THE MARKER BYTES SO THE PROGRAMS COULD RECOGNIZE THE ORDER OF THE BYTES IN THE FILE TO KNOW IF THEY ARE IN LITTLE ENDIAN OR IN BIG ENDIAN you can found a lot of information about it in the internet. but no one really tells you that this is necessary to make the file recognizeable by Excel as a csv file.
private static byte[] GetUCSStringFromData(string value)
{
    if (String.IsNullOrEmpty(value)) return new byte[] { };

    value = value.Replace("\",\"", "\"\t\"");

    var utf8Encoding = Encoding.UTF8;
    var utfBytes = utf8Encoding.GetBytes(value);

    var ucs2Encoding = Encoding.GetEncoding("utf-16le");
    var ucs2Bytes = Encoding.Convert(utf8Encoding, ucs2Encoding, utfBytes);

    var bytesFinal = new List {0xff, 0xfe};
    foreach (var b in ucs2Bytes)
    {
        bytesFinal.Add(b);
    }

    return bytesFinal.ToArray(); // ucs2Encoding.GetString(ucs2Bytes);

}

protected void ExportToCSVForExcel(object sender, EventArgs e)
{
    if (String.IsNullOrEmpty(CSVData.Value)) return;

    var strFileNameexport = FileNameHidden.Value.Replace(" ", "_");
    Response.Clear();
    Response.ClearContent();
    Response.Buffer = true;
    Response.ContentType = "text/comma-separated-values";

    //Those lines makes it possible to work with UCS2
    Response.ContentEncoding = Encoding.GetEncoding("utf-16le");
    Response.Charset = Response.ContentEncoding.HeaderName;

    var data = GetUCSStringFromData(CSVData.Value);

    Response.AddHeader("Content-Disposition", string.Format("attachment;Filename={0}.csv", strFileNameexport));
    
    Response.BinaryWrite(data);
    Response.End();
} 

So never forget it!

martes, 12 de mayo de 2009

Intellisense in NHibernate Mappings in Visual Studio 2008

I found this good post in StackOverflow, I put it here simply to have it always present.

To enable it, try copying the following files into: C:\Program Files\Microsoft Visual Studio 9.0\Common7\Packages\schemas\xml, or into C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas.

I'm not sure which is the right place and I put them into both of mine but it works for me.

  1. nhibernate-configuration.xsd
  2. nhibernate-generic.xsd (I could not find this file in the last version of the code of NHibernate
  3. nhibernate-mapping.xsd

You can get the files from the source of nHibernate

Taken from here

TableTypeString property of Dialect Class is never used in NHibernate

I was wondering why TableTypeString property of Dialect Class is never used inside the NHibernate library, the property is there, but when I try to find usages of this property, no usages were found.

I needed to use this property because I was writing a special Dialect for MySql to be able to generate InnoDb Tables and to make their charset UTF-8 by default in despite of the default charset of the Database.

I found an example of this in this post (Teamwork, MySQL and UTF-8), the code there is for Hibernate in Java, so instead of use the getTableTypeString getter in NHibernate there is a TableTypeString property.

So, I subclass the MySqlDialect to make my own Mysql5InnoDbUtf8Dialect, but it didn’t works. Looking at the code I found that the Table class in Hibernate differs from the NHibernate Class in the SqlCreateString method. At the end of that method we found in the Java class

if ( comment != null ) { 
	buf.append( dialect.getTableComment( comment ) ); 
}  
return buf.append( dialect.getTableTypeString() ).toString(); 

And at the end of the same method in the NHibernate class we found

if(string.IsNullOrEmpty(comment)==false)
{
	buf.Append(dialect.GetTableComment(comment));
}

return buf.ToString();

So as you can see if the TableTypeString property have to be used somewhere, this is the place, and currently is not used(If anyone knows why please, please let me know!)

so I make the change in the code, in the Table Class at the end of the SqlCreateString method (see code below)

if (string.IsNullOrEmpty(comment) == false)
{
	buf.Append(dialect.GetTableComment(comment));
}
			
return buf.Append(dialect.TableTypeString).ToString();
//return buf.ToString();

and recompile again Nhibernate. After that my new Dialect was working very Nice!

I'm not so sure if that is the best way to acomplish what I was needing to do (create tables with UTF-8 charset by default). Sure, if you have control of the database you can create the database with UTF-8 charset by deafult, but I really want to found a way to do it by code and not in the MySql Administrator.

Well let's work again...

jueves, 23 de abril de 2009

Dealing with Back Button and Secured Applications

Today during the development of my current assigned project, the client registered a really tricky bug. In our system we are using ASP.NET 2.0, and some times we do some ajax calls to improve the perfomance of the applications and to avoid been making postbacks when only a very few information really changes.

The problem with that approach was that when the user interacts with one of the pages that has ajax enabled behavior, and then navigates to a new page in the system and then click the back button the page was restored to its original state and the changes that were made using javascript were lost, since the page was served from cache and not for the Server.

The other problem related to the same issue was that once the user has been logged in, navigate to some pages and then logout, if he pressed the back button in the browser the pages where he was navigating were there and that was representing a potential security hole to the system.

in ASP.NET in the PageLoad Event of your page (the master page could be a better place to put it) you can try this

//Turn off cache
Response.Cache.SetCacheability(HttpCacheability.ServerAndNoCache);

And Please!!!!! make yourselves a favor and add this line!

Response.Cache.SetNoStore();

Sometimes the browser just ignores the Cache Directive.

I found a post that says that using the beforeunload event the page will be forced to reload the content from the server but it seems not to be working.

More information about this particular issue could be found here:

http://www.hunlock.com/blogs/Mastering_The_Back_Button_With_Javascript

viernes, 20 de febrero de 2009

Mysql UTF8 Encoding, and how to save Chinese/Japanese characters.

One of the requirements of the current application that I’m developing is to be able to save Unicode characters, but when we save the data to the db, the unicode characters were converted to a sequence of question symbols  (like these ??????) and that was really annoying.

Well we do some googling and found the answer here:

http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html

Well to make the story short, if you want to store unicode characters be sure that the my.ini (mysql.cnf in linux) has the following lines in it.

collation_server=utf8_unicode_ci
character_set_server=utf8
character_set_client=utf8

I don’t know why, but It seems than the Mysql Connector for .Net was sending the queries statements  as non unicode, so, when the data arrives to the db it arrives with the wrong encoding and it stores the question symbols instead of the unicode characters.

If you don’t have control of the mysql Server, maybe you could run the equivalent sql statements before the queries, something like this :

SET NAMES UTF-8

I don’t know how to do it when you use NHibernate, (because the sql statements are sent by the NHibernate layer), but I read some posts that said it works.

Hopefully it can helps someone else.

Important note: those lines should be added after the default-character-set to work properly

miércoles, 28 de enero de 2009

Steps to deploy an ASP.NET 2.0 web application in Apache 2 using mod_mono 2.2 (In OpenSuse 11)

this is a quick list of the steps required to deploy .Net applications under Apache 2 and mod_mono in OpenSuse 11

  1. Compile your web application. You can do it on Linux using MonoDevelop or using Visual Studio 2008, we usually compile it on Linux just to be sure the website will no have problems to start.
  2. Create a folder for your application.

    In our case we created a folder named "WebApp " under this directory:

    /usr/share/mono/asp.net/apps/

    just because there were other demo applications that come with mono.

    Be sure to give this folder the right permissions. open a Console and log in as the root user (sudo -s) and move to the folder. Use the chmod command to change the permissions if necessary (chmod 777 ./ will enable all the permissions under this folder, you can try adding only the necessary access level, because 777 enable all of them)

  3. copy the contents of the compiled web application to the new folder created.
  4. navigate to the folder "/etc/apache2/conf.d" and create a text file named "WebApp.conf"
  5. copy this code inside the file
        Alias /WebApp /usr/share/mono/asp.net/apps/WebApp
        MonoApplicationsConfigFile WebApp /etc/xsp/2.0/applications-available/WebApp.webapp
        MonoServerPath WebApp "/usr/bin/mod-mono-server2"
        MonoSetEnv WebApp MONO_IOMAP=all
        <Location /WebApp>
         Allow from all
         Order allow,deny
         SetHandler mono
         MonoSetServerAlias WebApp
        </Location>
    
  6. As you can see in the line 2 we point to a WebApp file that should be located in the /etc/xsp/2.0/applications-available/, so navigate to that folder an create a WebApp.webapp text fileand put this code inside this file:
    <?xml version="1.0" ?>
       <web-application>
        <name>WebApp</name>
        <vpath>/WebApp</vpath>
        <path>/usr/share/mono/asp.net/apps/WebApp</path>
        <enabled>true</enabled>
       </web-application>
    
  7. Now is time to restart apache, so run this command: "apache2ctl restart".
  8. Now open your browser and navigate to http://localhost/WebApp/Default.aspx (if Default.aspx exist or maybe other page that you have in the site)
  9. If all went without errors now you can enjoy your new website running under Apache 2 using mod_mono 2.2

Notes: There a lot of things that could go, wrong, maybe you will need to first Install Apache 2, if you don't already have it in your Linux distro, or maybe the mod_mono 2.2. Those tasks are out of the scope of this post. The present list of steps were written here just to never forget them.if you need help to install apache 2 or mod_mono, or even MONO 2.2, then you will need to do some googling by yourself. sorry :(

martes, 27 de enero de 2009

MONO Formview/ObjectDataSourceView Bug?. Why some of the sections of my website Dissapear?

UPDATE : MONO Has Fixed this Bug in the 2.4 release, at least it seems to be fixed looking at the source code

But it seems to fail when the result is null because it returns a new Object collection with a null element wich seems to be wrong, so this patch could still apply under some circunstances

Well some days ago, while we’re developing our current project release using MONO 1.9.1, MONO 2.0 were released. It was including a lot of bugfixes and new features. So now we were able to compile the application in Linux without make any modification. We were really happy.

But our happiness was futile. Suddenly we realize that some portions of the Website were  not been rendered. We review the rendered code and realized that the formviews were not been rendered properly.

The Gridviews and other ASP.NET controls were rendering good, but, not the formviews. As we had to continue with the project release, we decide to turn back to MONO 1.9.1. At the end of the past Sprint, we decide to rertun and investigate why the the Formviews were not rendering.

First we were suspecting that it could be maybe a Viewstate Issue, (When we turned off the viewstate for the formview controls in MONO 1.9.1 they were not rendered also, turning the viewstate on solved the issue). But turning on or off the viewstate in MONO 2.0.x had no effect. The formview were still not rendered.

So, after some time of digging into the code, creating our custom Formview controls that inherited the ASP.NET Formview, overriding some methods and using the Response as Console for debugging (God! Mono needs a Debugger!) we found the issue.

So, here is:

The problem is located at the ExecuteSelect Method of the ObjectDataSourceView Class that is located in this path inside the MONO Source directory:

mono-2.2\mcs\class\System.Web\System.Web.UI.WebControls\ObjectDataSourceView.cs

(The MONO sources can be obtained from this location)

the method has this lines of code that could be wrong:

if (result is IEnumerable)
return (IEnumerable) result;
else
return new object[] {};

As you can see when you have methods that return only a single instance of an entity, and those methods are used as the SelectMethods of the ObjectDataSource Controls, the ExecuteSelect will consider they’re not IEnumerable instances so it will return an Empty collection, causing that the Formview Controls render nothing at the Aspx pages.

the correct code should be:

if (result is IEnumerable)
return (IEnumerable) result;
return (result != null)? new object [] { result } : new object[] {};

as you can see, now only if the result is null will be returned an empty collection otherwise a collection with exactly one element is returned.

So doing that modification, and recompiling the MONO Sources you could make your Formview Controls work properly again.

I reported the bug, but I’m not so sure if that is really a bug, (because maybe it is intended to work like that in the specs…) or if it will be solved soon by the MONO team.

Well, here is some useful information.

Details of the Bug, and some comments about it:

http://go-mono.com/forums/#nabble-td21435835

To re compile MONO Sources, after do the code change:

http://blog.palehorse.net/2008/11/06/my-adventures-installing-mono-20-on-centos-4-to-work-with-apache-via-mod_mono/

Good Luck, Hopely this could help someone else!