An “instant performance boost saves hours on mission critical financial services operations” may sound like marketing collateral from a best case scenario.  However we are hearing this monthly from Fidelity Information Services (FIS) shops running the Advanced Commercial Banking System (ACBS) for loan processing.   Fidelity is certifying and supporting DataDirect drivers to improve End of Day batch processing performance against the Windows/Oracle distributed environment. While there are cases where our drivers improve performance up to 300% (or 1400% with DataDirect Bulk Load), we have consistently seen improvement on this particular ACBS function from 22-44% knocking off hours out of the box.  In addition, the driver has over a dozen connection properties to further improve Oracle performance based on your specific workload.

How to get started saving hours from your batch process

1. Download a trial of the DataDirect Connect for ODBC Oracle Wire Protocol drivers.  Note: ACBS requires 32-bit ODBC drivers even when running on a 64-bit Windows Server. Fidelity documentation refers to 6.1 drivers, however most shops choose to run the latest 7.1 version since it’s compatible.

2. Setup new ODBC data source (named ‘AGFDD’ in screenshot below for example) using DataDirect Connect for ODBC Oracle Wire Protocol driver.  Since DataDirect drivers communicate directly over the wire to Oracle, you can find the connection info (host, port, SID or service) in the client tnsnames.ora file; or configure the driver to lookup the connection directly from the tnsnames file.

3. Create an ACBS System Definition (Maintain Systems) to point to the new DataDirect DSN.  This separate ACBS System Definition must be defined and used when an End of Day job is submitted.  All other application functionality must use the ACBS System Definition tied to the native connection.

Lloyds TSB Bank in production with ACBS and DataDirect

Lloyds TSB Bank leverages Fidelity Information Services’ Advanced Commercial Banking System (ACBS) to automate and streamline loan processing. However, ODBC traffic accounts for the majority of  performance issues seen within ACBS batch environment. Performance monitoring highlighted excessive ODBC round trips. A typical batch run can incurs some 13 million ODBC round trips between the ACBS
application server and database. This increases to around 19 million at month end. The volume of traffic is typically 20GB requested by the client from the server with 5GB being sent to the server from the client. This increases at month end by around 40%.

Fidelity recommended and offered support for the DataDirect ODBC driver to Lloyds. Lloyds is now using DataDirect Connect for ODBC to connect to Oracle. On average we observed around 22% performance gain which reduces an average batch run by around an hour.

* This batch run already shows a significant improvement in runtime performance as a result of database cursor caching and SGA memory improvements under test in the QA environment at the time.

How are you guys so good?

Financial services shops understand the significant ROI from improving performance, and always ask how we do it and whether they can expect similar performance against DB2, Sybase and SQL Server.  The answer is “yes” thanks to our wire protocol technology.  We invest half our R&D into performance, including a performance lab that runs tests around the clock for typical data access scenarios ranging from transactional to batch processing.

While I love when shops ask how we’re so good, answering it makes me sound like a marketing puppet.  But feel free to ask again at 1-800-876-3101 and learn more about how ACBS shops are shaving hours from batch processing times.

My sentiment analysis research on the ODBC/JDBC application landscape produced some interesting results.  The rationale for this project was to leverage our customer data to help SaaS, BigData and other vendors predict patterns for data consumption and make the world a better place.  For example, I was talking to an ISV with both hosted and on-premise solutions for SMBs in healthcare.  In planning a data access strategy, we were able to advise on what infrastructure their user base is running to make adoption easy.

We are a data access company

Our view of the data connectivity landscape is unrivaled since our technology is vendor neutral and runs in virtually every organization across the globe.  The sentiment analysis discussed in this article is based on a count of unique direct end user account records with matching textual data and analyzed by industry, company size, and time over the last 10 years.  This data is not intended to compare and contrast the popularity of different solutions.

3 trends that surprised me

1. Desktop Application Market.  The number of shops looking to connect to sources from desktop applications grew 37.7% over the last 5 years including Microsoft Office, Data Visualization, Data Modeling, and Ad Hoc SQL Tools.  I found this surprising given the trend to move processing power to server and cloud.  I suspect this trend is being driven by the release of DataDirect Connect XE for ODBC Salesforce.com driver over the same period; and it will continue to grow with the release of DataDirect Cloud.

2. Healthcare Data Integration.  ETL Data Integration projects accounted for 31.37% of projects in the healthcare industry the last 5 years which is the highest of any vertical.  Perhaps this is expected given the shift to electronic health records in the industry.  Laura Madsen and Dr. Nemani Rao published an article concluding that, “ETL is critical to the success of any data warehouse project, particularly for healthcare, because so many transformations are needed to ensure that the data is usable for end users.”

Additionally, I will be hosting Jim Kosloskey, MD Anderson Cancer Center at Progress Exchange in Boston Oct 6-9 to learn about his experiences with data integration in healthcare.

3. Financial Services Intelligence. 40.28% of projects in financial services across Banking, Capital Markets, and Insurance were related to enterprise Business Intelligence platforms.  At first, I was surprised at the large percentage; but in drilling down to the details; different business units within the same bank  were often running different intelligence platforms.  In response to this trend, we published a white paper on Connectivity Solutions for Financial Services.

Sample set of applications considered in the research: Ab Initio, DataStage, Informatica, SQL Server Integration Services (SSIS), SAP Data Services, Oracle Data Integrator, Teradata Parallel Transporter, Talend, Pentaho Data Integrator, Business Objects, Cognos, OBIEE, Microstrategy, Pentaho, JasperSoft, SSRS, Tableau, Qlikview, Spotfire, Crystal Reports, SQL Server Linked Server, Oracle Database Gateway, Sybase ECDA, Microsoft Excel, Microsoft Access, Unica Campaign Affinium, Toad, DB Visualizer, CA Erwin, WinSQL, Aqua Data Studio, SAS, SPSS

Want to learn more?

DataDirect is the worldwide leader in data connectivity and trusted advisor for your next project.  Call us at 1-800-876-3101 and learn more about our unrivaled expertise in your industry’s application landscape.

To follow-up on my previous article on Teradata Parallel Transporter (TPT), I invited Jack Harris from WhereScape to share his latest performance tuning tips with TPT and DataDirect Connect for ODBC drivers certified for use by Teradata.  I have worked with Jack on several projects for WhereScape Red shops running Teradata TPT with DataDirect connectivity to build/renovate data warehouses and marts.

Jack’s contribution is specific to the DataDirect Connect for ODBC Oracle Wire Protocol driver, however ODBC performance tuning properties are available across all drivers used in TPT projects including SQL Server, Sybase, DB2, MySQL, Postgres, Salesforce, Informix, Hadoop Hive, and Cloudera Impala.

I’ve seen the Wherescape Wizard/Magician perform at different conferences and turn $1 bills into $100s right before my eyes; but I’ve got to say I’m even more impressed with Jack’s performance results.

Introducing Jack Howard from WhereScape:

One of the most exciting enhancements to Teradata’s TPT ODBC Operator was released in Version 14.0, efix 5.   While DataBlockSize may not cause you to wake up your wife in the middle of the night to make this announcement, it happened at my house.  The ODBC Operator is a vital member in a Teradata data loading strategy because data is never landed – among other issues, you can provision data directly into the database without concern for those pesky Carriage Return or Line Feed Characters one may encounter loading data from a file.

At the core of TPT’s architecture are Operators, and the behavior of an operator is modified by one or more Attributes.  The ability to add Attributes enables TPT to provide new features more easily than any other data provisioning technology available today.

Prior to the availability of the DataBlockSize attribute in the ODBC Operator, buffers were consumed in 64KB blocks, regardless of the size provided by the ODBC driver.  This had an adverse impact on performance – which in turn lead to a variety of perceptions, some of which were not founded in fact.

With the DataBlockSize attribute, you may now change the size of the blocks being read by the ODBC Operator to match the size of the block provided by the driver.   With the Oracle ODBC Wire Protocol driver, ArraySize is provided in bytes.  The DataBlockSize parameter reads kilobytes, so be sure to make the conversion to match these up correctly.

The TPT release note from Teradata provides some additional detail:

This optional attribute that allows the user to dynamically adjust the data block size of the buffer that will hold multiple rows with a single fetch call.  Additionally, the size of the row will affect the number of rows that can be fetched into that data block. End-users and customers will now have the ability to fine tune ODBC Operator performance by directly assigning a number value to the attribute.

In testing I’ve done with our customers, elapsed time has been reduced by over 80%.  This kind of improvement is exactly why you wake up your household, regardless of the time of day.  But, we know whenever a bottle neck is removed, new constraints to performance will become visible.

While the Oracle ArraySize parameter can be set to 4,294,967,296 bytes, we’ve found that probably won’t lead to optimal throughput.  Here are some rules of thumb to keep in mind:

• We’ve yet to see a benefit of ArraySize & DataBlockSize being configured beyond 2048KB with the Oracle Wire Protocol driver.  In most cases, a size between 1024KB and 2048KB has been optimal.
• This isn’t magic – if you have extremely large tables you need to ensure the SQL request to the database is supported with Where clause predicates that engage partitions, indexes, or other methods that can enhance database throughput.  Do some test queries in your favorite query tool to make sure you understand the access plan to the source data.  If “Select * From YourTable” doesn’t return in your query tool, don’t expect anything different.
• Think about setting QueryTimeout – this parameter will tell the ODBC driver to not timeout if the value is set to -1.  My suggestion is to set this value only after it’s proven to be needed.

Jack Howard is a Principal Architect for Teradata Solutions, WhereScape USA.

Bio: Data Architect focused on data warehouse solutions for the last twenty years, on Teradata.  In addition to Teradata, I’ve provided design, development and strategy services on Oracle, DB2, SQL Server architectures for the last thirty years at a variety of Fortune 500 companies.  My last position prior to joining WhereScape was a Center of Excellence Architect for the Teradata Corporation.

Does it sound like a dream? We see a lot people coming to us at DataDirect looking for improvement of Microsoft Access connectivity. Because even with much more sophisticated tools available, Microsoft Access is still beloved by many as an internal Business Intelligence tool and information storage. Unfortunately, the native driver has its limitations, and with different versions the problems vary.

Sumit Sarkar has covered two areas where we can help in previous posts:

Link or Import Salesforce data to MS Access in real-time

MS Access connectivity to Oracle databases, especially when looking for 64bit driver

So when should you look at DataDirect drivers?

You need easy integration with SQL databases:

Every version of Microsoft Access has different issues with connectivity. The latest, MS Access 2013, is missing SQL Server Upsizing wizard. You can try your best looking around, or rely on our driver to do the job.

When upgrading between different versions of MS Access and Windows:

Again – due to various versioning, one set up that worked with MS Access 2007 may not work with MS Access 2010, though you may need to upgrade to MS Access 2010 as your company is moving towards Windows 7. It can become a Catch-22.

You need a more robust driver:

As the title states, we’re talking stress-free connectivity to various data sources. DataDirect ODBC drivers are very robust, even if you need to import items such as text files, for example. Simply check our connectivity matrix to be sure your data source is supported. Most likely you will need DataDirect Connect for ODBC 32bit (to connect to Oracle, SQL server, DB2 and others) or DataDirect Connect XE for ODBC 32bit  (to connect to Salesforce and others), or if you have a newer version of MS Access in the 64bit mode, DataDirect Connect for ODBC 64bit or DataDirect Connect XE for ODBC 64bit.

You want your driver to support multiple database versions:

Knowing that setting up Microsoft Access correctly can be an issue, the DataDirect Technical support team has prepared plenty of “workarounds” available at our Knowledgebase. Simply look for “MS Access workarounds” or by error message.

You need better performance:

MS Access on Windows 7 can become very slow. Again, you can overcome the issue faced by native drivers with our ODBC driver. DataDirect drivers are industry standards for performance.

So what’s the bottom line? DataDirect ODBC drivers can really make your life much simpler, especially when you need to run and support multiple versions of MS Access.

What else can DataDirect do for MS Access data?

If you have MS Access databases and want integration of that data into your business intelligence and data warehousing infrastructure, we have solutions using DataDirect SequeLink to integrate Unix/Linux applications such as SAS, IBM Cognos, SAP Data Services, Business Objects, SSIS, or Ab Initio, to name a few. Or you can federate your data within Oracle using the database gateway for ODBC to MS Access or SQL Server Linked Server.

If you are interested how we could help you in your particular case, just try to download our free 15-day trial here.

Any questions? Contact us here or at Twitter @DataDirect_News. We are always happy to help you.

It’s hard to believe that it’s been almost three whole months since we released our ODBC driver for Cloudera Impala. We were excited to offer the expanded support, and we’re thrilled to continue to bring SQL access and standard based connectivity to our numerous users across geographies.

If you missed our webinar, “Powering Big Data Analytics with Cloudera Impala,” you can replay the recording here. So far, the reviews from the field are very good. Tested with Cloudera Impala 1.0 – it simply works.

Why Use Cloudera Impala?

The challenge of Big Data lies in how to analyze it efficiently. Cloudera Impala enables you real-time queries in Apache Hadoop. It scales across Hadoop cluster to support massive queries and executes queries in minutes and seconds, instead of hours and days. All this helps in being able to query your Big Data efficiently and truly opens it up to use across the whole organization.

What Do We Bring in?

• High performance connectivity
  • High throughput and low, more efficient memory usage

• ODBC standards compliance
  • Works with third party applications – you can get the SQL access you need!
  • Plug it in and it “just works”

• No client library to compile and distribute
  • Ease of deployment and ongoing management

• Multiple platform support
  • All major platforms
  • 64-bit support!

If you want us to help you to unlock the potential of your Big Data, you can simply try our driver trial here.

Any questions? Contact us here or on Twitter @DataDirect_News. We are always happy to help you.

People are looking for a Business Intelligence solution that will allow them to quickly analyze data. Most analysts don’t care where the data is stored – they care about getting the right results fast. BI solutions providers are making their best to make it happen for their customers, without bothering them too much about the technical details.

But behind every successful Business Intelligence solution is a piece of middleware that makes it happen. That’s were DataDirect steps in with our ODBC and JDBC drivers.

Why ISVs choose us?

It is often neglected that the bottleneck for quick access to your data is hidden in the middleware. It’s one of the reasons why our ISVs leave it up to us. Our drivers are robust, we are even an industry standard for performance! Couple this with excellent customer support (94% customer satisfaction), chuck in application fail over, versioning and bulk load, and it’s no surprise that over 400 Software Vendors embed our drivers, which include the top 10 enterprise software vendors in the world. We could keep on bragging and say that 98 of Fortune 100 companies are our customers, but of course, we would never do that.

See why Trillium Software chose Progress Software

Real-time Business analytics for everyone….

But what about the business analysts I talked about in the beginning? If the native driver is not robust enough for any reason, you can always opt for a third party driver, such as ours. Especially if your company has various data sources and different database versions. We make things simply work. No wonder we have the only ODBC driver for SQL Server recommended by IBM Cognos.

How can we help?

We have been in the middleware industry for over 20 years. We help our customers, both direct and indirect, by:

-        Increasing application performance for real-time analysis of various data sources

-        Supporting various database vendors, including versioning

-        Offering SQL access to non-SQL data sources (Salesforce.com, Hadoop Hive, Cloudera Impala)

-        Providing multiple platform support

If you want to see how we could help you to access your data, you can simply try our trial driver here.

Any questions? Contact us, ask us here or at @DataDirect_News. We are always happy to help.

I gave the prezi below on SQL access to SaaS at #PRGS13 in Boston and it was well received by Application Partners, Industry Analysts and Global Organizations in attendance.

The goal of the presentation at an appdev conference like #PRGS13 was two fold:

1. Illustrate how direct real-time SQL access instantly connects the business systems to SaaS data silos for data driven organizations.

2. Show ISVs how ODBC/JDBC access enables those data driven organizations to increase value from their SaaS applications. Salesforce platform is just one example.

What to think about while flying through the clouds

• What are the challenge of accessing SaaS data in the cloud?
• Where are the business systems and end users that should be consuming SaaS data through standard data access interfaces, ODBC and JDBC
• Observe the amazing technology that enables seamless SQL access to SaaS data

Related articles for SaaS ISVs

Ranking the ODBC/JDBC application landscape for SaaS vendors

Is your data access strategy blocking adoption of your SaaS application?

Getting started

Get more information on SQL access to Salesforce.com; visit www.datadirectcloud.com; or call us at 1-800-876-3101 to speak with a Systems Engineer to learn how to make data access to SaaS sources easy via ODBC and JDBC.

To share this post on Twitter, click here: http://clicktotweet.com/ bC1fV

After a dozen conversations with shops about mass data updates to the Salesforce platform, I decided to tackle this basic topic being called “one of the hardest easy things to do” by those who know SQL. The information in this article can be applied to INSERTS,DELETES,UPSERTS, etc; but I’ll focus on UPDATES to keep things simple.

Why UPDATES are the hardest easy thing to do?

Salesforce.com has an API which is very nice for force.com developers to manipulate data using APEX, but what about the rest who know SQL?  While SOQL is a subset of SQL, it supports DML such as updates very differently than SQL.

Here is a write-up on best practices for the Salesforce Platform for writing apex code related to UPDATES.

• Best Practice #1: Bulkify your Code
• Best Practice #2: Avoid SOQL Queries or DML statements inside FOR Loops
• Best Practice #7: Use of the Limits Apex Methods to Avoid Hitting Governor Limits
• Best Practices #10: Avoid Hardcoding IDs

In other words, there are several things to consider before performing updates to the platform through code.

How did DataDirect ODBC/JDBC drivers make this universally easy?

DataDirect Connect XE for ODBC and JDBC drivers make Salesforce.com look and feel like a relational database, so you can use SQL (or SQL based tools).  Since we communicate through the Salesforce published web service APIs, there is a significant amount of magic built into the drivers to provide a relational database like experience.

Thanks to the drivers, the only question you really need to ask is, “how many records am I trying to update using a single operation?”.  If the number is very large, consider setting the connection property EnableBulkLoad=true.  The connection property name may be misleading, but EnableBulkLoad applies to any bulk operation the Salesforce.com API can support including, DELETE,INSERT,UPDATE.  Further, we support these operations, as well as UPSERT, through DataDirect bulk interfaces for ODBC and JDBC that provide expanded functionality.

Test drive for UPDATE - 16X faster with 11X fewer web service calls

Environment Details:

• Salesforce Enterprise Account
• Connect XE for JDBC Salesforce driver v5.1
• O/S: Windows 7
• Application: DB Visualizer

Steps:

• Connect to Salesforce.com from DBVisualizer using JDBC
• Execute the query against Opportunity table with 174965 rows: UPDATE OPPORTUNITY SET AMOUNT=AMOUNT*1.1
• Determine # of web service calls used for the session with following query: select WS_CALL_COUNT FROM INFORMATION_SCHEMA.SYSTEM_REMOTE_SESSIONS WHERE SESSION_ID = CURSESSIONID() AND SCHEMA = ‘SFORCE’
• Retest with by setting connection property, EnableBulkLoad=true

Note: Performance can be further tuned with several supporting connection properties.  Test results are independent of the world famous DataDirect performance lab, and it is recommended to perform testing in a production like environment.

Results:

#EnableBulkLoad=false (default property value that works great for updating fewer records at a time)

SQL> UPDATE OPPORTUNITY SET AMOUNT=AMOUNT*1.1

12:09:05 [UPDATE - 174965 row(s), 3090.588 secs] Command processed
… 1 statement(s) executed, 174965 row(s) affected, exec/fetch time: 3090.588/0.000 sec [1 successful, 0 warnings, 0 errors]

Web Service Calls: 964

Time: 3090.588 secs

#EnableBulkLoad=true

SQL> UPDATE OPPORTUNITY SET AMOUNT=AMOUNT*1.1

13:32:05 [UPDATE - 174965 row(s), 186.319 secs] Command processed
… 1 statement(s) executed, 174965 row(s) affected, exec/fetch time: 186.319/0.000 sec [1 successful, 0 warnings, 0 errors]

Web Service Calls: 89

Time: 186.319 secs

Go for a SQL test drive against the Salesforce Platform

1. Download free 15 day trial of the DataDirect Connect XE for ODBC or JDBC Salesforce drivers to write SQL directly against Salesforce Platform.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more.

With the proliferation of data silos in SaaS applications, a lot of data driven organizations and application developers are getting headaches.

Meet Jeff

Jeff Leinbach is a Senior Software Engineer for DataDirect Cloud. He has been with Progress DataDirect for 9 years and worked on relational database drivers including Teradata, MySQL, DB2, Informix, Sybase and SQL Server.  Then the clouds rolled in and he transitioned to building connectivity to SaaS data sources such as:

• Salesforce Platform
• Oracle RightNow
• Microsoft Dynamics CRM
• Progress RollBase
• Eloqua
• Hubspot

He is a father, husband and big fan of the NC State Wolfpack.

Jeff gets a lot of headaches

Building standards based data connectivity to multiple SaaS sources requires many development hours since each SaaS data source is very different.  Each has a proprietary web service API across SOAP, REST, or both.  For example, Salesforce.com uses a SOAP API for fetching data and a REST like API for bulk operations.

Jeff is part of an army of highly skilled and talented engineers focused on standards based data connectivity for SaaS sources and each exhibits different types of aches.

To focus on Jeff, below are some things that contribute to his headache when building new sources:

1. Does the SaaS source have a query language?

Some SaaS sources have a query language that is typically a subset of SQL or “SQL like”.  For example, Salesforce Platform has SOQL (often pronounced “soakel” or “sockle”) while Oracle RightNow has ROQL (pronounced “rockwell”).

2. Great, a query language – now what?

Even if the source has a query language, each has its own rules for accessing data.  For example, some object models give information about relationships, but not all query languages support leveraging those relationships and even have different query syntax to associate data between multiple objects. In some cases, the query language can handle a particular query; and in others cases it cannot which means Jeff and team take care of it.

3. No query language?

For some SaaS vendors, it may not make sense to have a query language.  In those cases, building standard’s based connectivity can be equally challenging since each object is exposed with a different API with each having unique rules for invoking, filtering, searching, etc.

4. Let’s assume you can get the data back from the SaaS API

Each SaaS API formats results in a different way (JSON,XML,CSV,etc) making it hard to build a generic solution.  For example, each API has different governors or limits for fetching, aggregating and sorting results.  For example, some sources return a max of 10,000 results at a time, and then the data access code has to manually page each set and reorder the entire thing.   And not all objects are exposed in the same way.  For example, each API and corresponding filter rules can be different.

5. CRUD operations for SaaS Applications?

Updates, Deletes and Inserts with SaaS APIs are very different than relational databases.  For example, the API may require an ID be fetched for each target object to be updated.  This might be inefficient for updating multiple objects which may need to be chained to optimize network packets or use a different BULK API depending on what is available.  For more specific details, here is a write-up on how ODBC/JDBC drivers make UPDATES in Salesforce easy.

6. Handling Date Formats in the cloud

All SaaS dates are exchanged in UTC.  Not all clients expect dates in UTC; and it’s up to the data access code to localize the date appropriately, and ensure the round trip is successful for writes.  Then, a framework had to be developed to provide a standard way for consumers to format those dates and extract things like DAY, WEEK, MONTH, etc.

7. User Authentication

Each SaaS application has different challenges in authenticating users.  Jeff had too big of a headache to even expand further on this.

8. Is Metadata Static or Dynamic?

SaaS applications support either static or dynamic object models.  With dynamic object models, the schema needs to be discovered by the data access code. Some detect changes to the schema, and others do not.  I asked Jeff if they’re all different, and he replied emphatically, ”YES”.

9. Predicting performance between SaaS sources

Once you get the hang of building SaaS data connectivity, you are guaranteed to completely build connectivity and then run into a source that just won’t perform well under a certain workload sending you back to the drawing board to revisit all available APIs, architectures, and advil (aka the three A’s).

10. Maintaining matrix of changing SaaS APIs

SaaS applications are great since they make updates in the cloud seamlessly to the users.  Or at least, it appears seamless to end users; but to those building data connectivity, this can be the opposite of seamless (seamful?).  For example, Salesforce.com changes their API once a quarter which means your code needs to be recompiled each time.  And the metadata will likely change all over again (see above).  Further code changes are typical for new or modified interfaces.  Finally, Jeff and team need to certify each update before sending it out the door to users.

Don’t be Jeff

Visit www.datadirectcloud.com

Don’t tell Jeff, but his team’s hard work lets YOU focus on building really cool and amazing data driven applications INSTEAD of writing data access code that comes with the above headaches.

Dreamforce and SQL access made me almost famous

My session sold out last year to a packed house at Moscone, and attendees hit the ground running post DF12 on amazing data driven projects incorporating the Salesforce Platform.  Since the session, I have been invited to numerous conferences, meetups and user’s group from coast to coast to talk direct SQL access to Salesforce via ODBC and JDBC.  And I have hosted several interviews with industry analysts, technical journals and professional groups.

The response has been great, and I even get recognized on the streets wearing Progress DataDirect gear as the “Salesforce SQL blogger” for interviews including last month during Data Week in New York City.

And even my dog was interviewed in San Diego following his youtube video demonstrating SQL access to his pee app built on force.com.  And he was invited to attend Dreamforce 13 with a badge waiting for him at Moscone.

Dreamforce 13 and SQL access will make YOU really famous

Join me at Dreamforce for an encore presentation. They gave me a bigger room and upgraded time slot; so I’m looking forward to sharing everything I know to make you look good back home; and take control of Salesforce Platform data with direct SQL access.

Real-time SQL Access to Your Salesforce.com Data Using Progress Data Direct
SQL access to your salesforce.com data enables standards-based data access for business intelligence, data warehousing, and ad-hoc reporting and analysis. Join us to learn how to achieve this with real-time SQL-92 access, using ODBC and JDBC. We’ll show examples of organizations using SQL access, and you’ll see how you can get even more value out of your business data with these techniques.

Time & Location
Moscone Center West, 2009 | Tuesday, November 19th: 5:15 PM – 6:00 PM

Click here to see if there is an open session seat.

What will you get out of the session specifically?

• Learn how direct SQL-92 access to Salesforce.com works.
• Learn what data driven Salesforce shops are doing with SQL access today.
• See series of demos illustrating immediate business value across data visualization, data federation and SQL analysis including a really cool way to do updates.

Then walk down the red carpet

1. Download free 15 day trial of the DataDirect Connect XE for ODBC or JDBC Salesforce drivers to write SQL directly against Salesforce Platform.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more.

Latest data driven blogging from Dreamforce

• Preview on how the real-time SQL session will make YOU famous.
• Data related predictions from the week at Dreamforce.
• See how Progress Sofware leads the world in enabling use of Salesforce data, including a new one – art.   Our approach to Salesforce data was recognized as a top 50 amazing visual at Dreamforce which was pretty cool!

Real-time SQL access session Replay

Real-time SQL Access to Your Salesforce.com Data Using Progress Data Direct

SQL access to your salesforce.com data enables standards-based data access for business intelligence, data warehousing, and ad-hoc reporting and analysis. Join us to learn how to achieve this with real-time SQL-92 access, using ODBC and JDBC. We’ll show examples of organizations using SQL access, and you’ll see how you can get even more value out of your business data with these techniques.

Appropriate choice of shoe for the devzone

Session highlights:

• Roar of the group discussion was muted at: 2:25
• How it works: 6:00
• Use Cases: 8:33
• Real-time Data Visualization demo 11:07
• Real-time Data Federation demo 16:50
• Real-time Data Manipulation demo 22:18
• Takeaways 27:50

Is that Chilly registering?

The data mining dog “Chilly” who built intelligence using SQL from force.com had a full conference pass waiting for him thanks to my developer evangelist and conference hosts!

Too good to be true?

1. Download free 15 day trial of the DataDirect Connect XE for ODBC or JDBC Salesforce drivers to democratize your Salesforce Platform data.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more.

Interested in using PHP on various platforms including Linux? Looking into integrating your website or application scripted in PHP with various data sources? You may want to check out our Connect for ODBC drivers.

The common problem faced by PHP developers is how to make a scalable and fast connection to various data sources. In today’s world, everyone uses the magic words: real-time integration. Using a standard-based third party driver can simplify the whole process, saving you money and developer time. If you choose a driver that is easy to white label, your customer doesn’t even need to know.

An Example: Connecting SQL Server on Linux via ODBC

Let’s say you are looking for a 64bit ODBC driver for SQL Server on Linux for a custom made application written in PHP. You need an easy solution to improve connectivity between the application and SQL server. Usually this means rebuilding the PHP code to establish the connection, but you already spent significant man-hours on the project, and who knows how many customers actually need the connection?

Well, an easy answer is that DataDirect Connect for ODBC provides connectivity from UNIX/Linux platforms to the Microsoft SQL Server. Since our drivers offer wire protocol solutions, you don’t need database vendor client libraries. So in short, not only is this a scalable solution, it will also be very fast one (I do think you have heard about the bulk load before…).

Just Imagine the Possibilities…

PHP is a robust scripting language useful for everything from small projects to enterprise application development. After all, the PHP motto is: PHP powers everything from your blog to the most popular websites in the world. By using DataDirect Connect for ODBC drivers, you can easily write portable PHP code for all major operating systems and platforms. DataDirect Connect for ODBC offers both performance and scalability benefits through the use of its unique wire protocol architecture. Wire protocol architecture also removes the need for vendor client libraries, which results in easier deployment and maintenance of your applications.

Oh, last but not least, DataDirect Connect for ODBC drivers are the most widely used third-party ODBC drivers in the software market today and are backed by an award-winning technical support team.

A Few Practical Points

Use PHP function odbc_connect()to establish a connection to a database. This function accepts three required and one optional argument:

resource odbc_connect (string dsn, string user, string password [, int cursor_type]) U S I N G P H P

The dsn argument must be a Data Source Name. The user and password arguments identify the user. Finally, the optional cursor_type argument is used to specify, if necessary, which cursor method to use. For more information about this argument and PHP ODBC functions, consult the PHP documentation.

On Windows, the ODBC Administrator, available through the Control Panel, provides a GUI for creating a data source that stores connection option values. On UNIX and Linux, you store data sources in a file called odbc.ini. Instructions for defining a data source can be found in the DataDirect Connect and Connect XE for ODBC User’s Guide and Reference.

If, for example, you created a data source named DataDirectMSSQL for connecting to a Microsoft SQL Server database, and had a user name of sa and a password admin, you would use the following syntax in PHP:

$conn=odbc_connect(“DataDirectMSSQL”, “sa”, “admin”);

The $conn variable is a resource handle used to refer to this connection in subsequent code.

Intrigued?

We have prepared detailed tutorials for you on how to connect ODBC drivers on Unix/Linux or Windows.

You can also try Connect for ODBC for free with our 15-day trial. Or if you prefer to have a quick chat first, contact us on our website or on Twitter at @DataDirect_News. We are always happy to help you.

Following my third Dreamforce presentation on SQL data connectivity to Salesforce, I’m hearing more projects loading data into the Salesforce Platform.  The sales ecosystem at Salesforce continues to grow and shifting data gravity to the platform for sales operations.  With this shift, it’s making more and more sense to move data into the platform.

Why are shops moving data into Salesforce with SQL access?

• Salesforce Native Reporting for external data
• Data access via SOQL/APEX from force.com apps
• Build cloud data marts to consolidate on premise RDBMS which is possible thanks to Progress DataDirect Connect for ODBC and JDBC drivers that expose the platform as a relational database.
• On-premise Data Replication to the cloud to leverage unused capacity

Once the data is loaded into the platform, shops are leveraging the same DataDirect ODBC drivers to perform tasks such as real-time SQL data federation using SQL Server Linked Server and even some self service data connectivity using MS Access against Salesforce.  While this all works seamlessly, there is a lot of work that went into making the driver efficiently write data into the platform.

Getting started loading data into SFDC using SSIS

Note: Screenshots are for SQL Server 2008 R2.  Similar steps apply to SQL Server 2012.

1. Download trial of both the 32-bit and 64-bit DataDirect Connect and Connect64 XE for ODBC Salesforce drivers.

2. Create a 32-bit and 64-bit ODBC System DSN to Salesforce.com with the SAME name (use shortcut from driver program group to load corresponding 32-bit or 64-bit ODBC Administrator).  I used the name “Salesforce” as an example in screenshot below and specified a location to write the local schema files in the “Database” property to write to C:\ with file prefix “sumit.sandbox2″.

3. If you’re using SSIS 2012 or higher and inserting large numbers of records, check the “Enable Bulk Load” box on the bulk tab of the ODBC data source to transparently leverage Salesforce.com’s BULK API for optimal insertion.  Large means “a few hundred thousand records” per Salesforce’s “Best Practices for Deployments with Large Data Volumes“.

4. Create a new connection manager using the Provider, .NET Provider\ODBC Data Provider using your Salesforce login and test the connection.

5. Create new ADO.NET destination and select the Connection manager configured for Salesforce.  For SSIS 2012, there are native ODBC sources and destinations which are pretty cool when you upgrade.

6. Map your source and destination columns.  Note: The driver supports CREATE TABLE operations against Salesforce to create objects and fields in case you’re using replication or SQL tasks.

7. Run the package, green is good!

How do you get instant access to on-premise systems from Salesforce?

That is not possible today with Salesforce.  However, DataDirect Cloud connectivity is being leveraged by RollBase for an amazing data driven aPaaS experience providing instant access to on-premise data, as well as Salesforce.com.

It’s all about the data connectivity

1. Download free 15 day trial of the DataDirect Connect XE for ODBC Salesforce drivers to load your data into the Salesforce Platform.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more about our thoughts on data connectivity and aPaaS.

Following my widely popular tutorial for Business Objects Universe to Salesforce produced with SAP, I’m hearing several requests for real-time ODBC3 access to complementary data such as marketing automation in Eloqua, Marketo or Hubspot.  In response to the feedback from numerous BOBJ shops and from BOBJ User Groups/Conferences – we are rapidly delivering these SaaS data sources for Business Objects shops via DataDirect Cloud.

Why love the Business Objects Universe to Marketing Automation

• Marketing data is now sexy and multi source universe (MSU) support with BO 4.0 enables real-time data federation across sales data in Salesforce or Microsoft Dynamics with marketing data in Marketo, Eloqua and Hubspot.
• This solution leverages existing skills and your investment in SAP.   And Business Objects professionals are awesome, so try and network with them around the water cooler whenever possible.
• Hit the ground running with DataDirect ODBC3 connectivity to Marketo, Eloqua and Hubspot built from the same client codebase as the trusted relational connectivity available from Business Objects.
• Everyone is asking for it including the Eloqua topliners community, and ODBC3 access from DataDirect has been the unanimous choice for real-time Universe access to SaaS APIs. 
• DataDirect connectivity has the most secure architecture for accessing SaaS data from Business Objects.  Most recently, we passed an IT security review consisting of 80 questions in flying colors.

Getting started with BOBJ Universe to Eloqua

1. Sign up and log-in to your DataDirect Cloud account.

(In the spirit of Universe technology, I want to emphasize that DataDirect Cloud is a real-time data service)

2. Select your marketing automation data source such as Eloqua, Marketo or Hubspot.  In this case, I am selecting Eloqua since I have a production account.

3. Configure your connection information to Eloqua.  My Eloqua admin setup an account and I received an e-mail from support@eloqua.com with the connection information.

I set user name, password and company using the custom properties as follows: “Company=Progress;FailOnIncompleteData=1″.

Note that the screen may change since this was taken during beta.

4. Download and install the “Progress DataDirect Cloud Driver for ODBC” from DataDirect Cloud.  For Universe Design Tool (UDT) or Information Design Tool (IDT), you will need a 32-bit ODBC client.  For the Business Objects Application Server, you will need a 64-bit ODBC client.

5. Launch the ODBC Administrator from the “Progress DataDirect Cloud Driver for ODBC” program group and create a new data source.  The Database Name corresponds to the name configured in step 3.  You will login using your DataDirect Cloud ID from the ODBC administrator.

6- Launch your design tool (IDT or UDT), and configure the Generic ODBC 3 data source as type, “ODBC3″.

7- Enter User name, Password and Data source name as “Eloqua” in this example.

8- Click Next and continue to design your Universe to Eloqua similar to any other database.

Enjoy the all access data pass from your Universe

If you want to add MSU connectivity to SaaS data sources such as Salesforce, Microsoft Dynamics, Marketo or Hubspot: simply login to your DataDirect Cloud account and create a new connection.  Then create a new local ODBC data source using the same Progress DataDirect Cloud Driver for ODBC  that points to the new connection.

Since these are standard ODBC compliant connections, you can also use them from Data Services running on either Windows or Unix/Linux.

Call 1-800-876-3101 to speak with a systems engineer and learn how other organizations are making progress after creating a BOBJ Universe to marketing automation data.

There is a growing sentiment among data connectivity professionals, in the real world, around disruption.  From my consulting projects, it typically starts out with a requirement to integrate a new system and we deliver the data connectivity.

Next, we have a follow-up call with an Infrastructure Team Lead, Data Architect, or CDO/CIO; and they start telling me about the importance of standardization on DataDirect Data Connectivity.  I initially thought these were prank calls by our marketing folks that say the same thing, but in reality it’s just a sign of our business becoming more relevant than ever before.

Let there be disruption

It’s true, Progress Software is also guilty of disruption with our new cloud platform, Progress Pacific, that includes aPaaS.  But we’re also guilty of disrupting our own disruption (which makes everyone happy) by providing real-time direct ODBC/JDBC SQL access to the applications built on the Rollbase aPaaS, as well as standards based data access to external sources – both directions made possible by DataDirect Cloud.

But it gets really ugly outside the zen of Progress Software.  We see data silos in SaaS applications (i.e. Salesforce,Dynamics,Maketo), NoSQL (i.e. MongoDB or Cassandra); IoT (i.e. machine data); NewSQL; Graph DBs; in-Memory DBs; and of course big data.  Each of these disruptive technologies brings tremendous potential and ROI if we can just avoid the disruption of data integration.

Enable the data professionals to disrupt the disruption

Let me share concrete examples related to disruptive technologies and data connectivity from real-world projects on the ground:

1. Application Platform As A Service (aPaaS) has arrived

I have worked on several projects connecting existing data infrastructure to aPaaS platforms such as Force.com and Rollbase.  One common use cases is for SAP Business Objects Universe to Salesforce.com data.  The existing SAP BI infrastructure expects real-time SQL access to an operational data store, however many aPaaS platforms create a data silo where the database is inaccessible behind a public or private cloud.  In fact we’ve been told numerous times, which I shared at Dreamforce, that DataDirect connectivity enables aPaaS adoption when organizations require operational intelligence and real-time data access.  The aPaaS market is growing, and standard data connectivity to SaaS/aPaaS from DataDirect is here to disrupt that disruption for your existing systems.

2. Disruptive new system getting dumped on your team

I’m hearing this from data architects, CDOs, or CIOs from verticals such as Financial Services to Telco.  The challenge is the variety of specialized data sources coming in such as IoT data in compressed formats, SaaS applications, RDBMS with machine data in Postgres or market data in Sybase, NoSQL or NewSQL databases running fraud detection, etc.  Rather than hire a staff of data scientists, java developers, curl developers, etc to access data; leaders in this space are partnering with DataDirect (and our partners) to put a SQL front end on the disparate data to hit the ground running with those disruptive technologies from end to end.

Standardization from a development standpoint has been solved via ODBC and JDBC (thanks in large part to DataDirect’s professional contributions).  Since SQL can still vary, one  approach has been to pick a database technology (so far Oracle or SQL Server); and connect to the disruptive world through DataDirect ODBC drivers.  The SQL Server Linked Server or Oracle Database Gateway technologies combined with DataDirect’s breadth of data sources provide that standard SQL front end to the disruption.

As requirements grow, DataDirect standards based connectivity is flexible for expansion outside the RDBMS technology layer for direct connectivity from ETL, data warehousing, BI, analytics, application development, etc.  It makes a great addition to corporate application stores and marketplaces.

3. ISVs with disruptive technologies

If you’re an ISV building a solution without direct SQL access to the data, please consider building an ODBC drivers for all my friends in the enterprise.  I really love how Netsuite provided ODBC access to their SaaS application; and how Progress Software provided standards based connectivity to Rollbase on the Progress Pacific aPaaS.  We’re already hearing from Rollbase shops that this clears adoption hurdles for data driven organizations running Microsoft Excel/Access and BI/Data Visualization Tools.

Disrupt your day to get started disrupting the disruption

1. Download free trial of the DataDirect Connect drivers or DataDirect Cloud Service to democratize your disruptive data.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more.  We promise to limit saying the increasingly overused word “disruption” to 3 times on the call.

Based on some recent projects, I am seeing a trend that continues to blur the lines between business intelligence (BI) and application development. It’s only fitting to blog about this topic since Progress Software is the worldwide leader in data connectivity and earned Gartner’s Visionary status for its application development platform.

My recent intelligent application consulting projects:

As the appdev/BI lines continue to blur, there is increased interest in querying unstructured data such as Apache SOLR.  To provide standards based connectivity to Solr, DataDirect provides self service rapid development capabilities to build a prototype in days.  This is achieved with DataDirect OpenAccess, and I will share my Apache Solr ODBC connector built last Saturday morning that took me less than an hour.

Less than an hour to ODBC and JDBC connectivity to Apache SOLR

1-      Get started with SOLR tutorial: http://lucene.apache.org/solr/4_6_0/tutorial.html (for test purposes, I launched jetty with Solr WAR from example folder as “java -jar start”)

2-      Indexed sample data in exampledocs: user:~/solr/example/exampledocs$ java -jar post.jar *.xml

3-      Verify query works: http://localhost:8983/solr/collection1/select?q=solr&wt=json

4-      Install and configure the OpenAccess SDK by following the OpenAccess SDK Installation Guide.

5-      Configure the DataDirect OpenAccess sample for Apache Solr.

6-      Once complete, below is sample query using Interactive SQL (ODBC):

ISQL> SELECT * FROM SOLRDOCS WHERE QUERY = ‘solr’
NAME ID FEATURES QUERY
Solr, the Enterprise Search Server SOLR1000 {id=SOLR1000, name=Solr,
 the Enterprise Search Server, manu=Apache Software Foundation, cat=[software, s
earch], features=[Advanced Full-Text Search Capabilities using Lucene, Optimized
 for High Volume Web Traffic, Standards Based Open Interfaces - XML and HTTP, Co
mprehensive HTML Administration Interfaces, Scalability - Efficient Replication
to other Solr Search Servers, Flexible and Adaptable with XML configuration and
Schema, Good unicode support: héllo (hello with an accent over the e)], price=0.
0, price_c=0,USD, popularity=10, inStock=true, incubationdate_dt=Mon Jan 16 19:0
0:00 EST 2006, _version_=1457970426419347456} *

Rows selected = 1

Solr data returned to Microsoft Excel using ODBC

What about importing data into Solr?

DataDirect has an amazing selection of JDBC drivers (relational data sources,SaaS sources such as Salesforce, Eloqua, Marketo, and aPaaS such as Rollbase, Force.com) if you’re looking to import data into your Solr server using the DataImportHandler JDBC interface for data imports.

Are you a data architect, project manager, or java developer with questions?

Call us at 1-800-876-3101 to learn more about how DataDirect OpenAccess can get you connected using ODBC or JDBC to Apache SOLR, as well as any web service API or unstructured data source.  This is one of those projects everyone at a data driven organization will want their hands on.

From working on data connectivity projects for a decade, I can say with certainty that hybrid cloud / on-premise environments have become the norm.  And I’m hearing requirements for secure data connectivity to data stores that reside on-premise behind a fire wall; or in a public cloud server running on Amazon.  Rather than open a database port, VPN, or SSH tunneling; you can expose that data using the DataDirect Cloud On-Premise Connector with real-time and direct ODBC/JDBC data connectivity.  Today, we introduced support for OpenEdge 10.2B to 11.3, and Oracle, SQL Server and DB2 are coming soon using the same proven approach.

User Tested, CIO Approved

The DataDirect Cloud On-Premise connector capability certainly raises questions for the on-premise world, especially in very large enterprises.  Last week, I demo’d the on-premise connector technology to Progress Software’s CIO, Dave Benson, and his team.  After addressing their security questions, the team saw the benefits with some current initiatives at Progress Software, and approved of the architecture. Progress IT recommends partnering IT with business sponsors for collaboration on data connectivity projects in case there are concerns around access to an IT managed database.

Prezi for ODBC/JDBC from SaaS/PaaS over HTTPS to on-premise OpenEdge, Oracle, SQL Server, DB2

Highlighting capabilities of DataDirect Cloud and Rollbase

The prezi for my Progress Communities Webinar is followed by two live demos.  First, I show the hosted version of Rollbase federating external objects directly from on-premise RDBMS, the ubiquitous OpenEdge sports database.  Then, we can built a chart of items in the hosted SaaS BI application, Explore Analytics.  Finally, to prove the direct real-time connectivity to on-premise; we update a federated record from Rollbase and re-run the charts in Explore Analytics to ensure the updated record is coming back.  At this point, attendees will be mesmerized by the data connectivity!

Updates to on-premise data from hosted Rollbase.

Get started connecting with DataDirect from Cloud to Cloud, Cloud to On-premise, or On-premise to Cloud

1- Start a trial of DataDirect Cloud today

2- Or call us at 1-800-876-3101 to learn more about our cloud to cloud, cloud to on-premise, on-premise to on-premise and on-premise to cloud.

Infosphere DataStage is one of the top data integration platforms from a data connectivity perspective since most sources and targets are fueled by DataDirect ODBC connectivity.  This includes our streaming bulk load (without landing flat files) functionality for faster loads into popular RDBMS such as SQL Server from Unix/Linux.

I have been working with an increasing number of DataStage shops looking for expanded connectivity to SaaS applications using flexible ODBC access direct to Salesforce, Veeva CRM, Eloqua, Oracle RightNow and Marketo.  DataDirect delivers SaaS data connectivity that is built from the same framework as relational ODBC drivers already bundled with DataStage today.

Why add DataDirect SaaS connectivity to your stage?

• We’re putting a SQL-92 ODBC compliant interface on SaaS applications so ETL developers can hit the ground running. You don’t even have to know how to spell “eloqua”.
• We have Informatica shops running ODBC drivers to access Salesforce SaaS data in production today with complete read-write access, including streaming bulk load support.
• Disruptive SaaS data sources are coming soon to a DataStage near you.  I’ve seen a couple projects where organizations are forced to use a kludgy architecture to consume SaaS data against your own best practices/data governance for consuming SaaS data.
• IT can take control of the SaaS landscape before business sponsors even know what to ask.  It’s really important to me that you use the best platform for the job without spinning up one-off satellite data integration systems.
• The same connectivity works with any standards based IBM application including IBM Cognos (Data Manager, BI, Transformer), IBM Cognos TM1, IBM SPSS, DB2 Federation Server, Websphere Application Server … as well as any standards based applications from Oracle, SAP, or Microsoft.

Getting Started unleashing the power of DataStage on SaaS

Below are steps to integrate additional DataDirect ODBC drivers with DataStage:

1. If you only care about Salesforce, install the DataDirect Connect XE for ODBC Salesforce driver. If you’re planning to add support for other SaaS sources in addition to Salesforce, install the DataDirect Cloud client.  Both provide real-time direct connections to Salesforce suitable for use with enterprise ETL platforms.
2. Verify test-connection works independently from DataStage by following the corresponding quick start guide with the download.
3. Integrate the ODBC shared library path (e.g. LIBPATH on AIX, SHLIB_PATH on HP-UX pa-risc, and LD_LIBRARY_PATH on Solaris, Linux and HP-UX Itanium) in <InformationServer>/DSEngine/dsenv file.  (Note: If running on Websphere, the shared library path will need to be configured from Websphere)
4. Specify the ODBC data source name in the uvodbc.config
5. Merge the odbc.ini data source into the existing file used by DataStage typically found in $DSHOME.  Below is a sample Salesforce DSN:
   [Salesforce]
   Driver=/opt/Progress/DataDirect/Connect64_for_ODBC_71/lib/ddsfrc27.so
   Description=DataDirect 7.1 Salesforce
   HostName=login.salesforce.com
   SecurityToken=<your security token>
   StmtCallLimit=0
   TransactionMode=1
   Database=/tmp/sfdc
6. Optional: The Connect XE Salesforce.com driver requires a JRE, and if DataStage is running with an older JRE version, the driver can be configured to use a standalone JRE independent from DataStage by following these steps:
   - Configure shared library path with desired JRE. For example on AIX: - LIBPATH=/usr/java6_64/jre/lib/ppc64:/usr/java6_64/jre/lib/ppc64/classic:$LIBPATH
   - Add following property to Salesforce ODBC DSN in odbc.ini file: SQLEngineMode=1
   - Check whether the standalone java service is running, ps -ef | grep “sforce”
   - If not running, please start it from the following directory: /opt/Progress/DataDirect/Connect64_for_ODBC_71/java/lib
   - java -cp sforce.jar com.ddtek.cloud.sql.Server &
7. Restart the node agents and begin importing SaaS metadata via ODBC

Information Server Designer Client ODBC connection to SaaS

Learn more about the expanded Data Connectivity Options available from DataDirect

1. Download free 15 day trial of the DataDirect Connect64 XE for ODBC Salesforce drivers or try the DataDirect Cloud Connectivity Service to integrate an unlimited number of applications with SaaS data sources.

2. Or call 1-800-876-3101 to speak with a systems engineer to learn how other organizations are making progress with DataDirect Connect Salesforce integration.

With the proliferation of data silos in SaaS applications, a lot of data driven organizations and application developers are getting headaches.

Meet Jeff

Jeff Leinbach is a Senior Software Engineer for DataDirect Cloud. He has been with Progress DataDirect for 9 years and worked on relational database drivers including Teradata, MySQL, DB2, Informix, Sybase and SQL Server.  Then the clouds rolled in and he transitioned to building connectivity to SaaS data sources such as:

• Salesforce Platform
• Oracle RightNow
• Microsoft Dynamics CRM
• Progress RollBase
• Eloqua
• Hubspot

He is a father, husband and big fan of the NC State Wolfpack.

Jeff gets a lot of headaches

Building standards based data connectivity to multiple SaaS sources requires many development hours since each SaaS data source is very different.  Each has a proprietary web service API across SOAP, REST, or both.  For example, Salesforce.com uses a SOAP API for fetching data and a REST like API for bulk operations.

Jeff is part of an army of highly skilled and talented engineers focused on standards based data connectivity for SaaS sources and each exhibits different types of aches.

To focus on Jeff, below are some things that contribute to his headache when building new sources:

1. Does the SaaS source have a query language?

Some SaaS sources have a query language that is typically a subset of SQL or “SQL like”.  For example, Salesforce Platform has SOQL (often pronounced “soakel” or “sockle”) while Oracle RightNow has ROQL (pronounced “rockwell”).

2. Great, a query language – now what?

Even if the source has a query language, each has its own rules for accessing data.  For example, some object models give information about relationships, but not all query languages support leveraging those relationships and even have different query syntax to associate data between multiple objects. In some cases, the query language can handle a particular query; and in others cases it cannot which means Jeff and team take care of it.

3. No query language?

For some SaaS vendors, it may not make sense to have a query language.  In those cases, building standard’s based connectivity can be equally challenging since each object is exposed with a different API with each having unique rules for invoking, filtering, searching, etc.

4. Let’s assume you can get the data back from the SaaS API

Each SaaS API formats results in a different way (JSON,XML,CSV,etc) making it hard to build a generic solution.  For example, each API has different governors or limits for fetching, aggregating and sorting results.  For example, some sources return a max of 10,000 results at a time, and then the data access code has to manually page each set and reorder the entire thing.   And not all objects are exposed in the same way.  For example, each API and corresponding filter rules can be different.

5. CRUD operations for SaaS Applications?

Updates, Deletes and Inserts with SaaS APIs are very different than relational databases.  For example, the API may require an ID be fetched for each target object to be updated.  This might be inefficient for updating multiple objects which may need to be chained to optimize network packets or use a different BULK API depending on what is available.  For more specific details, here is a write-up on how ODBC/JDBC drivers make UPDATES in Salesforce easy.

6. Handling Date Formats in the cloud

All SaaS dates are exchanged in UTC.  Not all clients expect dates in UTC; and it’s up to the data access code to localize the date appropriately, and ensure the round trip is successful for writes.  Then, a framework had to be developed to provide a standard way for consumers to format those dates and extract things like DAY, WEEK, MONTH, etc.

7. User Authentication

Each SaaS application has different challenges in authenticating users.  Jeff had too big of a headache to even expand further on this.

8. Is Metadata Static or Dynamic?

SaaS applications support either static or dynamic object models.  With dynamic object models, the schema needs to be discovered by the data access code. Some detect changes to the schema, and others do not.  I asked Jeff if they’re all different, and he replied emphatically, ”YES”.

9. Predicting performance between SaaS sources

Once you get the hang of building SaaS data connectivity, you are guaranteed to completely build connectivity and then run into a source that just won’t perform well under a certain workload sending you back to the drawing board to revisit all available APIs, architectures, and advil (aka the three A’s).

10. Maintaining matrix of changing SaaS APIs

SaaS applications are great since they make updates in the cloud seamlessly to the users.  Or at least, it appears seamless to end users; but to those building data connectivity, this can be the opposite of seamless (seamful?).  For example, Salesforce.com changes their API once a quarter which means your code needs to be recompiled each time.  And the metadata will likely change all over again (see above).  Further code changes are typical for new or modified interfaces.  Finally, Jeff and team need to certify each update before sending it out the door to users.

Don’t be Jeff

Visit www.datadirectcloud.com

Don’t tell Jeff, but his team’s hard work lets YOU focus on building really cool and amazing data driven applications INSTEAD of writing data access code that comes with the above headaches.

Dreamforce and SQL access made me almost famous

My session sold out last year to a packed house at Moscone, and attendees hit the ground running post DF12 on amazing data driven projects incorporating the Salesforce Platform.  Since the session, I have been invited to numerous conferences, meetups and user’s group from coast to coast to talk direct SQL access to Salesforce via ODBC and JDBC.  And I have hosted several interviews with industry analysts, technical journals and professional groups.

The response has been great, and I even get recognized on the streets wearing Progress DataDirect gear as the “Salesforce SQL blogger” for interviews including last month during Data Week in New York City.

And even my dog was interviewed in San Diego following his youtube video demonstrating SQL access to his pee app built on force.com.  And he was invited to attend Dreamforce 13 with a badge waiting for him at Moscone.

Dreamforce 13 and SQL access will make YOU really famous

Join me at Dreamforce for an encore presentation. They gave me a bigger room and upgraded time slot; so I’m looking forward to sharing everything I know to make you look good back home; and take control of Salesforce Platform data with direct SQL access.

Real-time SQL Access to Your Salesforce.com Data Using Progress Data Direct
SQL access to your salesforce.com data enables standards-based data access for business intelligence, data warehousing, and ad-hoc reporting and analysis. Join us to learn how to achieve this with real-time SQL-92 access, using ODBC and JDBC. We’ll show examples of organizations using SQL access, and you’ll see how you can get even more value out of your business data with these techniques.

Time & Location
Moscone Center West, 2009 | Tuesday, November 19th: 5:15 PM – 6:00 PM

Click here to see if there is an open session seat.

What will you get out of the session specifically?

• Learn how direct SQL-92 access to Salesforce.com works.
• Learn what data driven Salesforce shops are doing with SQL access today.
• See series of demos illustrating immediate business value across data visualization, data federation and SQL analysis including a really cool way to do updates.

Then walk down the red carpet

1. Download free 15 day trial of the DataDirect Connect XE for ODBC or JDBC Salesforce drivers to write SQL directly against Salesforce Platform.

2. Or call 1-800-876-3101 to speak live with a Systems Engineer to learn more.