IN Software,Thoughts

2 wishes ,2 useful things contd..

2006-02-12T20:56:00.000-08:00

Continuing my previous post ...There is one more thing that I find extremely is needed :a way to create empty directory structures in your system, what I mean is something by which I can right click on a folder or a set of folders and say 'select to create directory structures' and wroom!!! You navigate to a specified directory and create empty directory structures of the selected source ..
Applications of the above would be say :you want to replicate a directory structure for a source code at different places with different source code files but maintain the same directory structure.

Another useful thing that you would need would be a facility that would help you copy a diff between the content two directories..A lots of times you would need to have the same content in two different directories .That's when this will be very useful..

Two wishes ..two useful things

2005-12-02T04:25:00.000-08:00

I need two facilities one in my web e-mail services(eg. Yahoo) and another on my destkop.On the first one:there are situations where in you have to send e-mails that have the same attachments(documents,pictures etc).
Cases could be:
1.I need to send my company information documents(which are not subjected to change regularly) to a client who requests it.
2.A job seeker post his resume to different companies but uses the same resume document each time to send.
In these cases although the attachments are same ,the recipients are different each time and the e-mail is send at different times .I get to know to who I have to send, each day and not before hand.For such situations,if the web e-mail service could provide me a facility where I can keep these set of documents on its server.And then send it for me ,when I request it to do so as an attachment with my e-mails, wouldn't it be great ?.Of course the storage space will have to be judiciously utilized for storing these attachments,but it should not be a constraint considering the fact that most commercial web e mail services provide 1 Gig Inboxes!! these days.. Lets see if any of the e-mails services start providing it...

The next thing I wish for..I will put in the next post.. :)

REST,SOA

2005-10-17T21:51:00.000-07:00

REpresentational State Transfer is a new buzzword in the Integration world.It supports for a common set of interfaces and a only a limited few for all applications to use and varies from the SOA in this very sense that ,in SOA the wsdl defines different port interfaces for different services types..
find more..
http://searchwebservices.techtarget.com/topic/0,295492,sid26_tax301569,00.html

Connecting USBs

2005-07-18T02:04:00.000-07:00

Today ,if I see, most electronic devices have a USB facility on them.I can connect my device to any computer system that can detect my device and connect the two systems for data transfer.But a negative that I observed is it is not possible to connect any two USB devices themselves and expect data to get transferred between them.
Now with the expounding of the USB,what I think has become necessary is that of a universal driver for USB devices that would enable any two USB devices to communicate and this is what exactly products from companies like 'Delkin Devices' for example do

'Delkin Devices'

Companies like these have come with 'USB bridges' which connect USB devices that store data in FAT16/FAT32 file system format.It will make the life of consumers easier in downloading song files from an USB storage device to an IPOD or storing pics from the camera onto a USB storage device and many more things..

Universal file storage format

2005-06-13T21:59:00.000-07:00

Haven't blogged recently but,I realize that I did not find anything worth writing.But a recent development does sound interesting,using XML to store Microsoft office format files.Its good since many of the earlier binary formats were too proprietary.Now other applications can also make sense of the office formats and finally be able to process office documents..still a long shot though as some glitches are still around..
For further reading...
Click here

Binary XML Storage

2005-05-13T01:47:00.000-07:00

A recent update on XML transmission has been the conversion of XML from being transmitted as text to a much optimized way of converting it to a compressed format and then transmitting('gzip' in this context is pretty well known) for reasons such as network bandwith since XML as text is pretty verbose.

What I have been imagining is why not always store XML in binary format and applications working on XML uncompress the XML.So much for saving storing space..but every 'bit' saved counts heee

Escalating

2005-04-26T22:16:00.000-07:00

On a recent visit to the US I observed an increasing use of escalators.What I also observed was the fact that at peak times the escalators were heavily used, while at other times the escalators simply kept moving..I thought, what a huge waste of energy.Sophistication does come at a price..the price of wasting energy.I wondered why couldnt they be more efficient and perform dual function of sophistication as well as optimization.Afterall pitfalls of wasting energy are alaraming,added with the fact that energy resources are fast depleting,an efficient use of all devices,machines that consume energy is a must,no matter what the source of energy is..remember energy saved at one place could be a used hungrily elsewhere.There is enough demand for energy everywhere.
What I was thinking on these lines was why couldnt sensors be placed on the escalator to detect if a person is standing on it or not.Just a simple thought which is quite possible with electronic sensing devices these days coming petty cheap :)

In the meanwhile I started another blog sometime back just to log thoughts more generic in nature,more to do do with life and its idiosyncrasy..check out the link below:
Musings of a Wandering Mind

Speeding XML processing

2005-02-14T20:33:00.000-08:00

A few days back, I read what I thought was going to come one day:Hardware for accelarating XML processing.Yes, with XML as the medium in alomost all network communication,clogging of the processing bandwith with XML was sure to come.It was inevitable,as parsers for XML(SAX or DOM)resulted in slow performance in routers ,switches and other such network elements where XML processing occurs significantly.

So the story today is that hardware will accelarate XML processing,the same way as Math Co-Processors one day took the onus of maths computing(although in this case number representation for different data also contributed, along with slow processing for the situation to have a co-processor).Altough only a few companies are out with the silicon for XML processing,many nore will surely follow.

I have been thinking of the possiblity wherein the chip stores ahead of time the XPath queries in the XML, the moment it is asked to process a certain XML.If it does this all it has to do is index through this already stored Xpath results when a request for a particular XPath query is made.This would result in faster retrieval of the requested data.This is of considerable advantage since most of the systems requiring XML processing are either Retrieval systems or Assertion systems.Of course for tranformations and XML modifications this will not have as much significance.It remains to be seen if we see any development in this area?

On Software Patterns

2005-01-16T22:41:00.000-08:00

Recently..I began to wonder about "Software Design Patterns".I began to wonder ,what are they based on?.Arent they afterall someone's experiences.For instance lets say we are talking about designs in civil engineering.The design, the engineer in that domain uses ,makes, are based on the solid foundations of mathematics.Lets assume that he has to construct a door,he will defintely follow a pattern based on usability and proper dimensions,he will know that the door has to be of 'x' feet high and 'y' feet wide(the maths of numbers again!!!).Proved and consistently proved mathematical formulas are the basis for the designer there, to determine what the dimensions of his structure are going to be.On the contrary when a software designer steps out to design something,what has he, to base his design on ?.Today I know that a code i had implemented long ago ,actually is a pattern..and I had done it without even having been aware of the fact that i had implemented the ADAPTER pattern.And to add to it,in most of the cases it has been found out that the design is more or less incomplete ,without its implementation in code bringing about some changes in the design again.And to think of it more than 60% of a project life cycle goes into planning and designing.So are we still very far from knowing what should be foundations of our designs ?.

Turning my thoughts elsewhere,I have been lately wondering of a particular use of Aspect oriented programming or Adaptive programming(call it AOP for simplicity).Its been a well known fact the features of a AOP could be used for logging et cetra..It is also a well realized fact that, database connections are a precious resource,and should be used as prudently as possible.In fact the shorter you hold the DB connection the better and the shortest we can try to hold a DB connection as programmers, could be at a method level.It is here that we could use the features of AOP to turn on the DB connection(get a hook to it) at the entry point of the method and release the same at the exit of the method.This sure follows a pattern,so we can as well, call this a pattern.
Links you may find useful :
Martin Fowlers ideas
IBM DeveloperWorks

Sample Event Brokerage

2005-01-02T20:55:00.000-08:00

Sometime back I believe ,I said that I would post some code that demoed the process of Event Brokerage.I am putting the same in this blog.Having been so much hands on in .net throughout,I could but possibly put the code ,only in C# :) . I assume many are familiar with the language derived from Java and C++.It has this feature of delegates that so well suits our needs.Delegates are much similar to function pointers in C++ in terms of functionality and help in implementing callbacks.Note how the publisher and the subscriber have been isolated in the code,this imlementation is based on the observer pattern(for more information on Observer Pattern design click here) .Much of the code has been explained in the commenting itself.It can also be easily followed of course!!!.The code requires at least version 1 of .net framework to execute.Here we go..........

using System;
using System.ComponentModel;
using System.Windows.Forms;
namespace Event
{

//dummy events to be used by the subscribers
public enum EventsToGenerate {NewEvent}
//delegate used to pass the callback in the subscriber
//to the event broker

public delegate void Notify(object sender,EventsToGenerate evt) ;

///

Class is an end appliction that subscribes for
/// an event and also invokes the publisher for publishing an event.
///

class RunEvent
{
///

The main entry point for the application.

[STAThread] static void Main(string[] args)
{
//subscribe for a particular event
EventConsumer consumer = new EventConsumer() ;
consumer.Subscribe(EventsToGenerate.NewEvent) ;

//now publish a particular event type and
//see if the consumer is called
EventGenerator generateEvent = new EventGenerator() ; generateEvent.Publish (EventsToGenerate.NewEvent) ;

}
}

///

descibes an user defined event

class Events : EventArgs
{
EventsToGenerate evt ;
public Events(EventsToGenerate evt):base()
{
this.evt = evt ;
}
public EventsToGenerate Evt
{
get
{
return this.evt ;
}
}
}

///

It is the event publisher

class EventGenerator
{
EventBroker broker = EventBroker.GetEventBroker() ;
public void Publish( EventsToGenerate evt)
{
broker.Delegate(Event.EventsToGenerate.NewEvent) ;
}
}

///

Defines the subscriber logic

class EventConsumer
{
public void Subscribe(EventsToGenerate evt)
{
if(evt == EventsToGenerate.NewEvent)
{
EventBroker broker = EventBroker.GetEventBroker() ;
Notify notify = new Notify(Handler) ;
broker.EventDistributor(notify) ;
}
}

///

callback function which will be called by the event broker when events of a
///particular type occur
///

/// this can be used to determine the source of the event
/// event that occurred
protected void Handler(object sender,EventsToGenerate evt)
{
if(evt == EventsToGenerate.NewEvent)
MessageBox.Show("Handled the NewEvent! My God!!!") ;
else
{
MessageBox.Show("Couldnt get the NewEvent! My God!!!") ;
}
}
}

///

class provides brokerage between the publishers and the subscribers
///

class EventBroker
{
private static EventBroker broker ;

///

singleton event broker

/// single instance of event broker
public static EventBroker GetEventBroker()
{
if(broker == null)
{
broker = new EventBroker() ;
return broker ;
}
else
return broker ;
}

public event Notify notifyConsumers ;

///

notify the subscribers

/// the event to be notified to the subscribers
public void Delegate(EventsToGenerate evt)
{
this.notifyConsumers(this,evt) ;
}
///

subscribers use this to register the callbacks
/// with the event broker so that the event broker
/// can notify the callbacks whenever events occur
///

///
public void EventDistributor(Notify callback)
{
notifyConsumers += new Notify(callback);
}
}
}
Voila!!!!!

Web Services - Starring SOAP

2004-12-08T05:50:00.000-08:00

As we know today..Services today are hot cakes and Web services are even more hotter.Reason being ,access to web services via the HTTP.With HTTP as the transport mechanism in most web browsers, web services are becoming even more attractive.Access to Web services is through standard defined ports, on which listen most of the web servers today.Security is easily provided by the firewalls and web servers too are equally wily.Thats why web services are becoming increasingly popular .They also offer:
1. Functional services(Take for example, I have an application that allows online share transactions for the users.But before I actually allow the user to transact the shares I need to authenticate him or her.For this the Org/Corp where the user is registered usually exposes the authentication system as a web service so that the 'share transaction' Application can utilize the same for authentication.)
2. Web services define implementation agnostic means of consuming services.That is the consumer of the service is hardly aware of the implementation details(you count the golden eggs..not on the hen laying it!!!) of the service.It may be written in Java C++,.net or any other lanaguage,it may be running on any OS platform too ,for that matter!!!.So tommorrow if the implementation of the web service changes, the code of the consumer doesnt break down as the interfaces remain same .
3.Since the world W3C has come out with a set of protocols for invocation of web services via WSDL ,discovery using UDDI etc there is quite an amount of standardization on this front.So it is safe to use web services.
4.Finally Web services make distributed applications working seamlessly a reality

So much for using the web services but ,what are the ways of invoking the web services as of today?.Usually the way its implemented these days is :a SOAP message is wrapped over a HTTP call to the Web service.A proxy class resides on the client side that creates a SOAP message and invokes it over HTTP.There are enough library APIs both in open source world as well as Microsoft.net that help acheive this.On the other side-A SOAP handler resides on the Web server where the call is made .It extracts out the method name to be invoked from the SOAP message and then calls the proper method using this information.Essentially Web services are stateless much like everything on the web,they are request-response mechanisms.Hence each time a request is made a new object is created and on this object proper method to be invoked is determined by the SOAP message .The parameters for this method are also passed through SOAP messages.This way of invoking a web service today also known as RPC style of method invocation is a convinent apporach.But it has its own pitfalls.For one,
A drawback using SOAP messages in this way is that the web services invocation become tightly coupled to the method that they invoke.Consider this follwing scenario:
A SOAP message usually has a header and a body.The header contains the method name and the body the parameters .The SOAP message contains other things like whether its a request message or a response message and so on..we will only concentrate on the coupling part.Look at the SOAP message format

< soap:Envelope >
< soap:Header > < method name="method-name" > func1 < /method >
< soap:Header >
< soap:Body >
< params >
< param1 > ... < /param1 >
.
.
. < param4 > ... < /param4 >
< /params >
< /soap:Body >
< /soap:Envelope >

The above XML is usually the structure of the SOAP mesage in the RPC style of method invocation.Imagine what happens when the implementor of the web service decides that for method 'func1' he will have 3 parameters from say the existing 4 parameters.This simple change will result in the code breaking up on the client side,because remember it has a proxy class that build the SOAP message for the method invocation and that class will now have to undergo a change.This becomes unacceptable espcically when the number of clients are in a huge number and live systems are running.Instead we would want something that can cause no code break on the client side.There is another style of method invocation called as Document style based over SOAP that can also be used for Web Service invocation.Its philosophy is simple-Web services are not just method invocations but a larger service that does a sizeable chunk of data prcocessing that would have otherwise caused the client to spend a sizeable amount of resources as well as time doing the same.Thats how the Web Services should be used too in theory.So instead the SOAP message in the document sytle becomes something like

< soap:Envelope >
< soap:Header > < method name="general-method-name" > GeneralMethod
< /general-method-name >
< /soap:Header >
< soap:Body >
<params >
< param1 > ... < /param1 >
.
.
. < paramN > ... < /paramN >
< /params >
< /soap:Body >
< /soap:Envelope>

The general method based on some more inputs from the client determines the particular method to be called and the parameters now become those of a generic nature without actually coupling the method and its parameters.The server does some more processing to determine what all parameters should go the particular method by parsing through generic parameters list.The actual internal method never and neednt get exposed to the clients this way.This obviously seems a better way of invoking the Web services.

Milking Event Brokers

2004-12-02T19:40:00.000-08:00

This blog dwells over the idea of event brokers comparing it with a real life sceanario.Most articles that I browsed on the Web were either too technical or were implementations; so I thought that I will start about something more conceptual and introductory than being specific.

Brokers are any agents acting in between 2 or more parties,more specific is the case of brokers in between 'producers' and 'consumers'.Lets call applications that produce the events henceforth as publishers and consumer applications henceforth as subscribers.

It is a well established fact that in an event driven system,paradigm of producers and consumers decoupled are prevalent.Producers and Consumers neednt be aware of each other or dependent on one another.All they may be interested is the produce that exchanges between them.How are brokers mirrored in our day to day life,although one may find many, a simple one is that of which follows.Imagine of the daily 'milkman' who delivers milk packets at your doorstep(I know its hard to imagine a milkman in the context of a software event broker,but read along...perhaps u will find the analogy interesting). Imagine for a minute,what would happen if that milkman was not there.Obviously we would ourselves have to go to the milk dairies for it(but remember we are interested only in getting the milk and not where it comes from.) .Imagine we having to go to the milk dairies everyday to buy milk,isnt that a burden ?.Its the same as the subscribers going and asking the the publishers explicitly for event notfications.This obviously in a distributed envirnoment, results in coupling between the systems.I will tell how;tommorrow if suddenly I decide to go for a different brand of milk ,I cant go to the same milk dairy again and ask for that brand of milk packets ,can I ?.Instead if I had common place ,where in I can inform that I need to be delivered milk packets of any brand that I would want,it would be great.On similar lines ,in distributed systems too..the same problem occurs.If I place my subscription logic in the publishers and tommorrow if I want to add/change the subscription to another event,I will have to add/remove the logic again in the publisher .On the side of the publishers also, this is disadvantageous as the publishers tend to be heavy with the subscriber logic in them ,when many subscribers are interested in being notified about the events it produces(Imagine at a milk dairy thousands of people crowded to get milk packets..it is logistically impossible!!!).Thats exactly where the 'event broker' fits in.

The job of the milk dairies is to produce milk packets and not bother about who uses them.Simlarly the consumers of the milk should only bother about getting the brand and quality of milk they are interested in.That is one should be independent of another.That being the case ,the same applies for applications that are interested in being notified about events,they neednt be aware of who the producer of the events are nor need to be aware of its intricacies..all they are interested is the events(milk packets in our case).This calls for decoupling between the applications and this is where the event broker plays a very important role.Event broker is the "milkman" or rather the 'middleman' who manages the event notifications.Just let the event broker know the events you are interested in and you will be notified about the same(like, let the milkman know what brand of milk packets you want and he will have them delivered at your doorstep).The only thing that needs to be done for event communications is that publishers need to tell the event broker what event they are publishing and subscribers need to tell what events they are interested in being notified about,thats it :) ,the chunk of the event communications will be then done by the event broker.Any change you require with event subscriptions ,just tell it to the event broker(its like telling your milkman[your event broker :) ] to deliver a different brand of milk from tommorrow) ..and voila your jobs done!!!.Its that easy

Event Brokers also offer some exciting features of QOS

When you are not at home ,the milkman may check once or twice again to see if you are there,if you are ,he will deliver your milk packets then.Its like having lifetime for events,the event broker can keep checking if the subscribers are up,within the life time of the event
Ensure delivery of events once or twice or as many desired .(Does your mikman dump all the milk packets at your doorstep?!!!!!,he will only deliver as many as would have ordered.).Its the same with Event Broker ,it will only deliver events that you would have subscriber for.
There may be many milkmen working for a milk dairy(similar to Load balancing,the events broker may be distributed among many servers for purposes of 'load balancing' and 'nearest to the delivery point' features)

This was briefly about some thoughts on how we could mirror event brokers in our day to day life and understand it better.

Will try to put some related code in the next blog :))

Happy blogging

2004-12-01T00:51:00.000-08:00

Happier Me

2004-11-30T22:44:00.000-08:00

Happier Me

IN Software,Thoughts

2004-11-29T22:21:00.000-08:00

This being my first blog ,let me take the name of Lord Ganesha, before before begining my blog efforts.I dont want to start with any technical stuff right in my first blog itself and send you to the castles of boredom right away.We have all the time in the world ahead of us for that ,but I thought, rather, lets look back. Being in Software development although for a very small time , makes me think back on its implications in our day to day life.Have we ever imagined how many softwares built and existing today affect our day to life.Come to think of it all of them do..there arent any that dont serve a purpose.So whats this evolution?,this journey from stone tools to 'soft tools', from stone spheres to 'Web spheres' ?.Yes ,software has evolved and is a part of us ,we accept that ,everybody does.
From the the days of feeding binary code to machines to the present era of using frameworks which provide ,memory management ,code and network security ,software has always been evolving to make life of people making and using it easy.Bygone are the days when we had to struggle for writing procedures in assembly languages,and cometh have the days when we are talking abstractions to the level of Objects,Components and now today Services.I would love to talk about each one of the these :Assembly level code to OOP and Services(we will be henceforth always be doing it :) ) ,but probably not in this introductory note.

Keep posting...