Data Ingestion for Enterprise Data Platforms

May 5, 2014

http://hortonworks.com/wp-content/uploads/2014/03/Oil-and-Gas-Ref-Arch.png

The Ingestion Box in the reference architecture is displayed as the smallest box.  However, this is the component that integrates with all the available data sources.  This tends to be among the most complex and time consuming task, but tends to be relegated to a lower priority which is a big mistake.

One needs to prioritize the data sources that generate maximum value and ensure we can ingest the data into the Big Data platform for subsequent “cool” analytics.

In my experience, it is also extremely important to have a robust User Interface for the ingestion section.  Otherwise, there could be a series of manual steps leading to errors and ingestion of “bad” data that will minimize impact of subsequent analytics.

 

 

Advertisements

Design First Followed by Microsoft too

October 13, 2013

I am total believer in the “Design-First” approach where we think about the UI from user perspective before we engage Technical architects.  I have led many such successful project over the past several years and believe it is a recipe for success. Looks like Microsoft has taken the same approach.  Check out http://gigaom.com/2013/10/13/a-peek-inside-microsofts-new-design-first-development-strategy/


Faster & Secure Web

April 15, 2013

you should definitely try disconnect.me   It makes your web browsing experience faster and I was amazed at the number of companies tracking my browser usage (Now blocked with disconnect.me)


Usability and Design gain market traction

March 19, 2013

Users want simple and effective interfaces.  Without a simple interface, the application is made hard to use and will be harder sell.

http://tinyurl.com/d5fwjh2

You know your web application design is minimalistic & Effective when design is such that when it is used in mobile devices, the interaction with your app is simple .

Effective design is an opportunity for entrepreneurs to apply good design principles to cumbersome applications and capture market share


Is Bootstrap cool or way cool?

January 17, 2013

Just released a 30 page site built with bootstrap.  Simple & extremely fast to build.  built in less than 1 week with  1 developer new to the bootstrap framework.   Site works very well on iPAD, Android OS and iPhones.

http://tinyurl.com/a8slw79

http://tinyurl.com/7faxdys


Software Project Success – Prototype First

November 13, 2012

As I review the successful (defined as on-time and within budget)  & unsuccessful projects that I have led over the past 10 years, a common thread has been that successful projects had at least 80% complete prototype prior to start of any development work.  This is spread across both Agile and waterfall methodologies.

The prototype allowed the Business Analysts to “touch” the product and better visualize & communicate what they expect of the product.

To get a project kicked off, start with a UI Developer and a Business Analyst.  Refine requirements till you are 80% complete and then bring Software Architects.

This may seem contrary to several Agile approaches, but in my perspective it is a modified Agile approach that has worked for my teams.  The agility is divided into 2 phases – Agile requirement gathering (with UI developer & business Analyst) and then move to Agile software development.

I have also noticed that bringing software developers into the process prior to the product vision being ready,  tends to compromise the product vision and it is best to involve them in architecture & development once the vision is clear.