Monitoring Hadoop MapReduce Applications

April 29, 2013

While users have access to many tools that assist in performing large scale data analysis tasks, understanding the performance characteristics of their parallel computations, such as MapReduce jobs, remains difficult.  Step #1 is to create a test suite that you can reliably run after every change.

http://www.myhowto.org/java/hadoop/2013/03/06/monitoring-performance-of-your-hadoop-mapreduce-application/


Faster & Secure Web

April 15, 2013

you should definitely try disconnect.me   It makes your web browsing experience faster and I was amazed at the number of companies tracking my browser usage (Now blocked with disconnect.me)


Lessons learnt in IT Offshore Management for Startups

April 12, 2013

No matter how brilliant your new software  may be, it’s doomed if you can’t figure out how to make it efficiently, consistently and economically.

Starts up engage offshore teams due to lower costs & ease of getting teams engaged with tools like e-lance.  However, Creating brilliant and functional software is more difficult than ever due to a lack of offshore team  management skills . My experience at managing offshore IT organizations has taught me some important lessons about software design and production that, if heeded by software startups, provide an opportunity to bring innovative products to market without suffering setbacks – or even failure – from preventable mistakes.

1 – Get Inside the Software Shop

I’ve met too many people in this game who spend several weeks researching software vendors, in some cases even identify top talent, hand over project details  and get on the phone once a week for a status report.  This leads to enormous problems at launch time – Badly designed product and a mismatch between expectation & software delivered.

Speak on a daily basis.  Speak directly to the individual developers, testers, Business Analysts and not just the project managers.  You’ll be amazed at what you learn about the software development process and your software development partner.  Seemingly small pieces of information from these conversations can later help you refine product design or even clue you in to larger issues with the partner’s management.

 

2 – Build Prototypes Closer to home

You often get what you pay for in this realm, so it isn’t where you want to pinch pennies. Use the prototyping phase to refine, refine and refine some more. That way, when it comes time to spend money on software development,   you only have to do it once.

The added benefit of prototyping close to home is that you can iterate faster.  Expedited turnaround times accelerate overall development cycles, and in turn, reduce development costs. Rapid development also gets your product to market faster than the competition!

 

3 – The job does not end after launch

Once you launch (congrats!), resist the temptation to sit back and watch it all happen. To the contrary, this is when real development begins.  Ensure you have left a good budget for further development.    I have seen far too many software startups spend money pre-launch, leaving no allotment for development activities once they have received feedback from real users.  This is where prototyping also helps.  Continue to meet with offshore partner on daily basis.

 

 


Should I use traditional HPC or Hadoop?

April 12, 2013

Excellent blog by Guident – http://tinyurl.com/bne6t8k  

comparing a  Hadoop versus Traditional High Performance Computing.  A specific use case of reading large log files is compared and Hadoop is the winner in terms of performance.

What happens if we have access to traditional HPC hardware?  Should we use Hadoop on HPC?   Check out excellent article by S.Krishnan on this – http://tinyurl.com/cwbzvof .   Results are not conclusive, but an interesting read.

Bottom line, it appears to be dependent on the use case?  Has anyone done a more detailed comparison?


Usability and Design gain market traction

March 19, 2013

Users want simple and effective interfaces.  Without a simple interface, the application is made hard to use and will be harder sell.

http://tinyurl.com/d5fwjh2

You know your web application design is minimalistic & Effective when design is such that when it is used in mobile devices, the interaction with your app is simple .

Effective design is an opportunity for entrepreneurs to apply good design principles to cumbersome applications and capture market share


Democratization of Social Media Data

March 18, 2013

The state of North Carolina has made available to the general public more than 100k  social media records from selected North Carolina state agencies.

It is a live customer deployment available for others to play around with.  Anyone interested in social media archiving should certainly check it out.

http://tinyurl.com/bp89sxm

 


My Startup Investment Philosophy

March 17, 2013

I recently started investing in Technology Startups in the Houston area.   I have developed a list of criteria important to me:

  1. Expertise to Guide – I would like to spend at least a few hours every week on the startups before I invest in the entrepreneur.  This would primarily be in guiding the founder & their team. My expertise is in delivering Enterprise SaaS to Enterprise customers.  This encompasses technologies related to Big Data, Cloud Computing, Machine Learning, and Predictive Analytics. Areas I can guide the entrepreneur include Technology selection,  Technology Strategy, Vendor Selection , Offshore Development Management, Software Delivery, Project execution and connections to other people who can assist the entrepreneur.  I am not interested in investing in startups led by entrepreneurs that don’t need or want my guidance.  I invest my money only after assisting the entrepreneur for several months to confirm that the entrepreneur has a need for my expertise.
  2. Action Orientation – The Entrepreneur should work fast, focused on securing customers developing the product in an iterative fashion.  I prefer speed of execution over a great idea.  Excuses from the entrepreneur for not moving fast is a red flag.
  3. Demonstrated Priority – Entrepreneur should be willing to sacrifice career to a good extent to accomplish Business goals. Priority is demonstrated by entrepreneur who puts money into his/her Business.  I need the entrepreneur to have put in at least $25,000 of his/her money in product development costs.  Prior to investing $25,000, the entrepreneur should not be seeking investment money from anybody.

Scaling Web Applications with NoSQL

March 16, 2013

The need to scale web applications in internet speed requires the auto-sharding and “no-schema” flexibility of NoSQL databases.

Good article from Couchbase describing need for NoSQL Databases  – http://tinyurl.com/4ek3j79


Future of Law Practice

March 12, 2013

Per the recent conference conducted by ReInvent Law Laboratory:

“It will be user-friendly and accessible via bright and fresh retail shops with the ambiance of Apple stores. It will be data-driven, with litigators turning to enormous databases capable of predicting results and guiding strategy. It will have the charm of an assembly line that parcels work out across time zones and specialties in structured processes certain to warm the hearts of project managers. And it will be beautiful. Imagine strings of case citations rendered as computer-generated graphics as appealing to the eye as they are to the analytical mind.”

Imaging walking into a Apple like store and getting legal services!!!


My favorite Information Governance Commandment

March 12, 2013

Check out Bhandari’s 10 commandments – http://tinyurl.com/aaxjehy

My favorite “To Have and to Hold is Good, but not til Death Us Do Part”

Recommendation is not to keep data indefinitely because storage is cheap. When you get into dispute the cheap storage suddenly becomes expensive since you might have to collect at source and have it reviewed by lawyers.   Vendors with Predictive coding technology have the foundation to build a solution for defensible deletion and reduce the volume at source.