Salesforce System Integration Interview Prep

Remote Process Invocation – Request and Reply

Best solutions

  1. External Services – invokes a REST API call – allows to invoke externally hosted service in declarative manner. External REST service are available in a an OpenAPI or Integrant schema definition. Contains primitive data types, nested objects not supported. Transaction invoked from Lightning Flow. Integration transaction doesn’t risk exceeding the synchronous Apex governor limits.
  2. Salesforce Lighting – consume WSDL and generate Apex proxy. Enable HTTP (REST) services, GET, PUT, POST, DELETE methods.
  3. Custom Visualforce page or button initiates Apex HTTP callout – user initiated action calls and Apex action that executed this proxy Apex class

Suboptimal

  1. Trigger – calls must be asynchronous (@future)
  2. Batch job callout – bundle responses together and make 1 callout for every 200 records processed.

Endpoint capability

Endpoint should be able to receive a web services call via HTTP. Salesforce must be able to access endpoint via the internet, curl or postman. Apex SOAP callout- WSDL 1.1, SOAP 1.1

Apex HTTP callout -REST service using standard GET, POST, PUT, DELETE methods

Data Volumes

Small volume, real time activities, due to small timeout values and maximum size of the request or response from Apex call solution.

Timeliness

  1. Request is typically invoked from user interface, must not keep user waiting
  2. Governor limit of 120 second timeout for callouts
  3. Remote process needs to be completed in SF limit and user expectations else seen as slow system
  4. Apex synchronous apex limits are 10 transaction than run for 5 seconds
    • Make sure external server callout is less than 5 seconds

State management

  1. Salesforce stores the external system External Id for specific record
  2. The remote system stores the Salesforce unique record ID or other unique key

Security

Any call to external service should maintain:

  1. Confidentiality
  2. Integrity
  3. Availability

Apex SOAP and HTTP callouts security considerations

  1. One way SSL is enabled by default
  2. 2 way SSL is enabled by self-signed certificate or CA-signed certificate
  3. WS-Security is not supported
  4. If necessary use one way hash or digital signature using Apex Crypto to ensure message integrity
  5. Remote system must be protected by appropriate firewall mechanism

Error Handling – error occurs in form of error http error codes

400 – Bad request, 401 – Unauthorized, 404 – Not Found, 500 – Internal server error

Recovery – changes not committed to Salesforce until successful response, retry

x amount of times until success received else log error.

Idempotent Design consideration – duplicate calls when something goes wrong needs

need to have a message ID to make sure duplicates are not created

Http h = new Http();
HttpRequest req = new HttpRequest();
req.setEndpoint(url); //
req.setMethod('GET'); //

//Basic Authentication - specify in Named Credentials so no code change needed
String username = 'myname';
String password = 'mypwd';

Blob headerValue = Blob.valueOf(username + ':' + password);
String authorizationHeader = 'Basic ' +
EncodingUtil.base64Encode(headerValue);
req.setHeader('Authorization', authorizationHeader);

HttpResponse response = h.send(req);

if (response.getStatusCode() == 200) {
    Map<String, Object> results = (Map<String, Object>) JSON.deserializeUntyped(response.getBody());
    List<Object> animals = (List<Object>) results.get('animals');
    System.debug('Received the following animals:');
    for (Object animal: animals) {
        System.debug(animal);
    }
}

//Implement httpCalloutMock
global class YourHttpCalloutMockImpl implements HttpCalloutMock {
    global HTTPResponse respond(HTTPRequest req) {
    

Test.setMock(HttpCalloutMock.class, new YourHttpCalloutMockImpl());

Remote Process Invocation – Fire and forget

Best solutions

  1. Process-driven platform events – using process builder/workflow rules to create process to publish insert or update event
  2. Event messages – the process of communicating changes and responding to them without writing complex logic. One or more subscribers can listen to the same event and carry out actions through CometD.
  3. Customized-driven platform events – events are created by Apex triggers or batch classes
  4. Workflow driven outbound messaging – remote process is invoked from an insert or update event. Salesforce provides workflow-driven outbound messaging capability that allow sending SOAP messages to remote systems.
  5. Outbound messaging and callbacks – callback helps mitigate out-of-sequence messaging also 2 things Idempotency and Retrieve of more data. When creating a new record and update externalId with the recordId from external system.

Suboptimal

  1. Lightning components – user interface based scenarios, must guarantee delivery of message in code
  2. Triggers – Apex triggers to perform automation based on records changes
  3. Batch jobs – calls to remote system can be performed from a batch job. Solution allows batch remote process execution and for processing of reposes from remote system.
MyEventObject__e newEvent = new MyEventObject__e();
newEvent.Name__c = 'Test Platform Event';
newEvent.Description__c = 'This is a test message';
List<Database.SaveResult> results = EventBus.publish(newEvent);

	
Test.startTest();
// Create test events
// ...
// Publish test events with EventBus.publish()
// ...
// Deliver test events
Test.getEventBus().deliver();
// Perform validation 
// ...
Test.stopTest();


//Listener - EMPConnector use login to and create EmpConnector(bayeuxParameters)
import static com.salesforce.emp.connector.LoginHelper.login;
//create consumer
SalesforceEventPayload eventPayload = new SalesforceEventPayload();
	Consumer<Map<String, Object>> consumer = event -> {
			eventPayload.setPayload(event);
			itsReqHandler.handleRequest(eventPayload.getPayload());
};

//add subscription to event
subscription = empConnector.subscribe("/event/" + EVENT_NAME, replayFrom, consumer).get(WAIT_TIME, TimeUnit.SECONDS);

Error handling and recovery

Error handling – because this pattern is asynchronous the callout of event publish handles need to be handles as well as the remote system needs to handle queuing, processing and error handling.

Recovery

More complex, need some retry strategy if no retry is receive in the QoS (Quality of Service) time period

Idempotent Design considerations

Platform events are published once, there is no retry on the Salesforce side. It is up to the ESB to request that the events be replayed. In a replay, the platform events reply ID remains the same and ESB can try to duplicate messages based on reply ID.

Unique Id for outbound messages is sent and needs to be tracked by the remote system to make sure duplicate messages or events are not repossessed.

Security

  1. Platform events – conforms to the security of the existing Salesforce org
  2. Outbound messages – One way SSL enabled, two way SSL can be used with outbound message certificate. Whitelist the IP ranges of the remote integration servers, remote server needs appropriate firewalls
  3. Apex callout – one way SSL enabled, 2 way SSL through self signed or CA signed certificates. Use one way hash or digital signature (Apex Crypto class) to ensure integrity of message. Remote system protected by appropriate firewall mechanisms.

Timeliness

Less important, control handed back to client immediate or after successful deliver message. Outbound message acknowledgment must occur in 24 hours (can be extended to seven days) otherwise message expires.

Platform events – send events to event bus and doesn’t wait for confirmation or acknowledgement from subscriber. If subscribe doesn’t pick up the message they can reply the event using replayID. High volume message are stored for 72 hours (3 days). Subscriber use CometD to subscribe to channel.

State management

Unique record identifiers are important for ongoing state tracking:

  1. Salesforce store remote system primary or unique surrogate key for remote record
  2. Remote system store Salesforce unique record ID or some unique surrogate key

Governor Limits

Limits depends on the type of outbound call and timing of the call

Reliable Messaging

  1. Platform Events – form of reliable messaging. Salesforce pushed the event to subscribers. If the message doesn’t gets picked up it can be replayed using reply ID.
  2. Apex callout – recommend that the remote system implement JMS, MQ however it doesn’t guarantee delivery to remote system, specific techniques such as processing positive acknowledgement from remote endpoint in addition to custom retry logic must be implemented.
  3. Outbound messaging – if no positive acknowledgment receive, retry up to 24 hours. Retry interval increase exponentially 15 sec to 60 min intervals.

Publisher and subscriber not in same transaction

Publish event before committed to the database, subscriber receives the event and does lookup to not find the record.

Publish behavior – publish immediately/publish after commit

Publish events – @future or @queueable to callout to events only when commit has complete

Message sequencing

Remote system discard message with duplicate message ID

Salesforce send RecordId, remote system makes callback to Salesforce

Handing Deletes

Salesforce workflow can’t track deletion of records, can’t call outbound message for deletion. Workaround:

  1. Create custom object called Deleted_Records__c
  2. Create trigger to store info (unique identifier) in custom object
  3. Implement workflow rule to initiate message based on the creation of custom object

Batch Data Synchronization

Best solutions

  1. Salesforce change data capture (Salesforce master)
  2. Replication via third party ETL Tool (Remote system master) – Bulk API
  3. Replication via third party ETL Tool (Salesforce master) – SOAP API getUpdated()

Suboptimal

  1. Remote call-in – call into SF, causes lots of traffic, error handling, locking
  2. Remote process invocation – call to remote system, causes traffic, error handing, locking

Extract and transform accounts, contacts, opportunities from current CRM to Salesforce (initial data load import)

Extract, transform, load billing data into Salesforce from remote system on weekly basis (ongoing)

Data master

Salesforce or Remote system

 

Salesforce Change Data Capture

Publish insert, update, delete, undelete events which represents changes to Salesforce. Receive near real time changes of records and sync to external data store.

Takes care of continuous synchronization part, needs integration app to receive events and perform update to external system

Channel – /data/{objectName}_Change

Error handling – Pattern is async remote system should handle message queuing, processing and error handling.

Recovery – initiate retry based on service quality of service requirement. Use replyID to reply stream of events. Use CometD to retrieve past messages up to 72 hours.

Bulk API – Replication via 3rd party ETL tool (more than 100 000 records)

Allow to run change data capture against source data. Tool reacts to change in the source data set, transforms the data and them call Salesforce Bulk API to issue DML statement, can also use SOAP API.

  1. Read control table to determine last time job ran, other control values needed
  2. Use above control values to filter query
  3. Apply predefine processing rules, validation rules, enrichments
  4. Use available connectors/transformation capability to create destination data set
  5. Write the data to Salesforce objects
  6. If processing is success update the control variable
  7. If process fail update control variable for process to restart

Consider

  1. Chain and sequence ETL jobs to provide cohesive process
  2. Use primary key for both systems to match incoming data
  3. Use specific API methods to extract only updated data
  4. Importing master-detail or lookup, consider using the parent key at the source to avoid locking. Group contacts for an account to be imported at the same time.
  5. Post-import processing should only process data selectively
  6. Disable Apex triggers, workflow and validation rules
  7. Use the defer calculations permission to defer sharing calculations until all data loaded

Error handling

If error occur during read operation, retry for errors. If errors repeat implement control tables/error tables in context of:

  1. Log the error
  2. Retry the read operation
  3. Terminate if successful
  4. Send a notification

Security

  1. Lightning Platform license with at least “API Only” user permission
  2. Standard encryption to keep password access secure
  3. Use HTTPS protocol

Timeliness

Take care to design the interface so all batch processes complete in a designated batch window

Loading batches during business hours not recommended

Global operations should run all batch processes at the same time

State management

Use surrogate key between two systems.

Standard optimistic locking occurs on platform and any updates made using the API require the user who is editing the record to initiate a refresh and initiate their transaction. Optimistic locking means:

  1. Salesforce doesn’t maintain state of record being edited
  2. Upon read, records time when data was extracted
  3. If user updated the record and before save checks if another user has updated
  4. System notified user update was made and use retrieve the latest version before updating

Middleware capabilities

Middleware tool that supports Bulk API

Supports the getUpdated() function – provide closest implementation to standard change data capture capability in Salesforce

Extracting data

Use the getUpdated() and getDeleted() SOAP API to sync an external system with Salesforce at intervals greater than 5 minutes. Use outbound messaging for more frequent syncing.

When querying can return more than an million results, consider the query capability of the Bulk API.

Remote Call-In

Best solutions

  1. SOAP API – Publish events, query data, CRUD. Synchronous API, waits until it receives a response. Generated WSDL – enterprise (strongly-typed), partner (loosely typed). Must have a valid login and obtain session to perform calls. Allows partial success if the records are marked with errors, also allows “all or nothing” behavior.
  2. REST API – Publish events, query data, CRUD. Synchronous API, waits until it receives a response. Lightweight and provides simple method for interacting with Salesforce. Must have a valid login and obtain session to perform calls. Allows partial success if the records are marked with errors, also allows “all or nothing” behavior. Output of one to use as input to next call.

Suboptimal

  1. Apex Web services – use when: full transaction support is required, custom logic needs to be applied before commenting.
  2. Apex REST services – lightweight implementation of REST services
  3. Bulk API – submitting a number of batches to query, update, upsert or delete large number of records.

Authentication

Salesforce supports SSL (Security Socket Layer) and TLS (Transport Later Security), Ciphers must be at least length 128 bits

Remote system has to authenticate before accessing any REST service. Remote system can use OAuth 2.0 or username/password. Client has to set the authorization HTTP header with the appropriate value.

Recommend client caches the session ID rather than creating a new session ID for every call.

Accessibility

Salesforce provides a REST API that remote system can use to:

  1. Query data in org
  2. Publish events to org
  3. CRUD of data
  4. Metadata

Synchronous API

After call to server it waits for a response, asynchronous call to Salesforce is not supported

REST vs SOAP

REST exposes resources as URI and uses HTTP verbs to define CRUD operations. Unlike SOAP, REST required no predefined contract, utilize XML and JSON for responses, and has loosely typed. Advantage includes ease of integration and great use for mobile and web apps.

Security

Client executing the REST needs a valid Salesforce login and obtain a access token. API respects the object and field level security for the logged in user

Transaction/Commit Behavior

By default every record is treated as a separate transaction and committed separately. Failure of one records does not cause rollback of other changes. Using the composite API makes a series of updates in one call.

Rest composite resource

Perform multiple operation in a single API call. Also use output of one call to be input of next call. All response bodies and HTTP statuses are returned in a single response body. The entire requires counts as single call towards API limit.

Bulk API

For bulk operations use Rest-based BULK API

Event driven architecture

Platform events are defined the same way as you define a Salesforce object. Publishing to event bus is same as inserting Salesforce record. Only create and insert is supported.

Error handling

All remote call-in methods or custom API require remote system to handle any subsequent errors such as timeouts and retries. Middleware can be used to provide logic for recovery and error handling

Recovery

A custom retry mechanism needs to be created if QoS requirements dictate it. Important to consider impotent design characteristic.

Timeliness

Session timeout – session timeout when no activity based on SF org session timeout

Query timeout – each query has a timeout limit of 120 seconds

Data volumes

CRUD – 200 records per time

Blob size – 2GB ContentVersion (Chatter)

Query – query(), queryMore() return 500 records, max 2000

State management

Salesforce stores remote system primary key or unique key for the remote record

The remote system stores the Salesforce ID unique record ID or some unique

Governor limits

5000 API calls per 24 hour

10 query cursors open at time

Reliable messaging

Resolve the issue that delivery of a message to remote system where the individual components may be unreliable. SOAP API and REST API are synchronous and don’t provide explicit support for any reliable messaging protocols.

Data visualization

Best solutions

  1. Salesforce Connect

Suboptimal

  1. 1. Request and reply – Salesforce web services APIs (SOAP or REST) to make ad-hoc data requests to access and update external system data

Access data from external sources along with Salesforce data. Pull data from legacy systems, SAP, Microsoft, Oracle in real time without making a copy of the data.

Salesforce connect maps data tables in external systems to external objects in your org. External objects are similar to custom objects, except they map to data located outside SF org. Uses live connection to external data to keep external objects up to date.

Salesforce connects lets you

  1. Query data in a external system
  2. CRUD data in external system
  3. Define relationships between external objects and standard or custom objects
  4. Enable Chatter feed on external object page for collaboration
  5. Run reports on external data

Salesforce Connect Adapters

OData adapter 2.0 or OData adapter 4.0connects data to exposed by any OData 2.0 or OData 4.0 producer

Cross-org adapter – connects to data that’s stored in another Salesforce org. Used the Lightning Platform REST API

Custom adapter created via Apex – develop own adapter with the Apex Connector framework

Calling mechanism

External Objects – maps SF external objects to data tables in external systems. Connect access the data on demand and in real time. Provides seamless integration with Lightning Platform can do global search, lookup relationships, record feeds.

Also available to Apex, SOSL, SOQL queries, Salesforce API, Metadata API, change sets and packages.

Error handling

Run Salesforce Connector Validator tool to run some common queries and notice error types and failure causes

Benefits

  1. Doesn’t consume data storage in SF
  2. Don’t have to worry about regular sync between systems
  3. Declarative setup can be setup quickly
  4. Users can access external data with same functionality
  5. Ability to do federated search
  6. Ability to run reports

Considerations

Impact reports performance

Security considerations

Adhere to Salesforce org-level security, use HTTPS connect to any remote system.

OData understand behaviors, limitations and recommendations for CSRF (Cross-Site Request Forgery)

Timeliness

Request invoked by user interface, should not keep the user waiting

May take long to relieve data from external system, SF configured 120 sec maximum timeout

Completion of remote process should execute in timely manner

Data volumes

Use mainly for small volume, real time activities, due to small timeout and maximum size of request or response for Apex call solution.

State management

Salesforce stores primary or unique surrogate key for the remote record

Remote system store SF unique record ID or other unique surrogate key

Microsoft Bing Adwords Java Integration

Step 1: Setup Bing BulkServiceManager Configuration

package com.integration.adwords.config;

import com.microsoft.bingads.*;
import com.microsoft.bingads.bulk.BulkServiceManager;
import com.microsoft.bingads.campaignmanagement.ICampaignManagementService;
import com.microsoft.bingads.internal.OAuthService;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.net.MalformedURLException;
import java.net.URL;

/**
 * Created by tmichels on 7/31/15.
 */
@Configuration
public class BingAdwordsConfig {

    @Value("${BING_ADS_CLIENTID}")
    private String bingAdsClientId;

    @Value("${BING_ADS_CLIENT_SECRET}")
    private String bingAdsClientSecret;

    @Value("${BING_ADS_ACCOUNTID}")
    private String bingAdsAccountId;

    @Value("${BING_ADS_CUSTOMERID}")
    private String bingAdsCustomerId;

    @Value("${BING_ADS_DEVELOPER_TOKEN}")
    private String bingAdsDeveloperToken;

    @Value("${BING_ADS_USERNAME}")
    private String bingAdsUsername;

    @Value("${BING_ADS_PASSWORD}")
    private String bingAdsPassword;

    @Value("${BING_ADS_REFRESH_TOKEN}")
    private String bingAdsRefreshToken;


    @Bean
    public OAuthDesktopMobileAuthCodeGrant oAuthWebAuthCodeGrant() throws MalformedURLException {
        OAuthDesktopMobileAuthCodeGrant oAuthWebAuthCodeGrant = new OAuthDesktopMobileAuthCodeGrant(bingAdsClientId, bingAdsRefreshToken);
        return oAuthWebAuthCodeGrant;
    }
    
    @Bean
    public AuthorizationData authorizationData(OAuthDesktopMobileAuthCodeGrant authentication){
        AuthorizationData authorizationData = new AuthorizationData();
        authorizationData.setAuthentication(authentication);
        authorizationData.setCustomerId(new Long(bingAdsCustomerId));
        authorizationData.setAccountId(new Long(bingAdsAccountId));
        authorizationData.setDeveloperToken(bingAdsDeveloperToken);
        return authorizationData;
    }

    @Bean
    public BulkServiceManager bulkServiceManager(AuthorizationData authorizationData){
        BulkServiceManager bulkServiceManager = new BulkServiceManager(authorizationData);
        bulkServiceManager.setStatusPollIntervalInMilliseconds(5000);
        return bulkServiceManager;
    }
}

Step 2: Setup Bing Adwords Service

package com.integration.adwords.service;

import com.microsoft.bingads.AuthorizationData;
import com.microsoft.bingads.bulk.*;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.concurrent.ExecutionException;

/**
 * Created by tmichels on 7/31/15.
 */

@Service
public class BingAdwordService {

    private static final Logger logger = org.slf4j.LoggerFactory.getLogger(BingAdwordService.class);

    private static final String OUTPUTFILENAME="bingAdwords.csv";
    private static final String FILEDIRECTORY="adwords/bing/";

    @Autowired
    BulkServiceManager bulkService;

    public void submitAndDownloadWithBulkServiceManager() throws IOException, URISyntaxException {
        try {
            logger.info("Start BingAdwordService submitAndDownloadWithBulkServiceManager");
            SubmitDownloadParameters submitDownloadParameters = new SubmitDownloadParameters();
            submitDownloadParameters.setCampaignIds(null);
            ArrayList<DataScope> dataScope = new ArrayList<DataScope>();
            dataScope.add(DataScope.ENTITY_DATA);
            submitDownloadParameters.setDataScope(dataScope);
            submitDownloadParameters.setFileType(DownloadFileType.CSV);
            submitDownloadParameters.setLastSyncTimeInUTC(null);
            ArrayList<BulkDownloadEntity> bulkDownloadEntities = new ArrayList<BulkDownloadEntity>();
            bulkDownloadEntities.add(BulkDownloadEntity.CAMPAIGNS);
            bulkDownloadEntities.add(BulkDownloadEntity.AD_GROUPS);
            bulkDownloadEntities.add(BulkDownloadEntity.KEYWORDS);
            bulkDownloadEntities.add(BulkDownloadEntity.ADS);
            submitDownloadParameters.setEntities(bulkDownloadEntities);

            BulkDownloadOperation bulkDownloadOperation = bulkService.submitDownloadAsync(submitDownloadParameters, null).get();

            BulkOperationStatus<DownloadStatus> downloadStatus;
            int waitTime = 5000;

            for (int i = 0; i < 24; i++) {
                Thread.sleep(waitTime);

                downloadStatus = bulkDownloadOperation.getStatusAsync(null).get();

                if (downloadStatus.getStatus() == DownloadStatus.COMPLETED) {
                    break;
                }
            }

            File resultFile = bulkDownloadOperation.downloadResultFileAsync(
                    new File(FILEDIRECTORY),
                    OUTPUTFILENAME,
                    true,
                    true,  // Set this value true if you want to overwrite the same file.
                    null).get();

            logger.info("Download BingAdwords file success %s\n", resultFile.getName());

        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (ExecutionException e) {
            e.printStackTrace();
        }
    }
}

Apache Camel Kafka Spring Integration

1. Configure Camel Kafka Component

package com.integration.camel.component;

import org.apache.camel.component.kafka.KafkaComponent;
import org.apache.camel.component.kafka.KafkaConfiguration;
import org.apache.camel.component.kafka.KafkaEndpoint;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by tmichels on 9/3/15.
 */

@Configuration
public class KafkaCamelComponent {
    
    @Bean
    public KafkaEndpoint kafkaEndpoint(){
        KafkaEndpoint kafkaEndpoint = new KafkaEndpoint();
        kafkaEndpoint.setZookeeperHost("localhost");
        kafkaEndpoint.setZookeeperPort(2181);
        kafkaEndpoint.setTopic("test");
        return kafkaEndpoint;
    }

    @Bean
    public KafkaComponent kafkaComponent(KafkaEndpoint kafkaEndpoint){
        KafkaComponent kafkaComponent = new KafkaComponent();
        kafkaComponent.setEndpointClass(kafkaEndpoint.getClass());
        return kafkaComponent;
    }
}

2. Configure Kafka Consume and Producer Route

package com.integration.camel.route;

import org.apache.camel.builder.RouteBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by tmichels on 9/3/15.
 */

@Configuration
public class KafkaCamelRoute {

    @Bean(name = "KafkaRouteProducer")
    public RouteBuilder kafkaRouteProducer() {
        return new RouteBuilder() {
            public void configure() {
                from("direct:kafkaRoute").to("kafka:localhost:9092?topic=test&zookeeperHost=localhost&zookeeperPort=2181&groupId=group1&serializerClass=kafka.serializer.StringEncoder").bean(kafkaOutputBean.class);
            }
        };
    }

    @Bean(name="KafkaRouteConsumer")
    public RouteBuilder kafkaRouteConsumer() {
        return new RouteBuilder() {
            public void configure() {
                from("kafka:localhost:9092?topic=test&zookeeperHost=localhost&zookeeperPort=2181&groupId=group1&serializerClass=kafka.serializer.StringEncoder").bean(kafkaOutputBean.class);
            }
        };
    }

    public static class kafkaOutputBean {
        public void printKafkaBody(String body) {
            System.out.println("KafkaBody result >>>>> " + body);
        }
    }
}

3. Start Kafka test by adding routes to CamelContext and starting

package com.integration.camel.route;

import com.integration.camel.component.KafkaCamelComponent;
import com.integration.camel.context.CamelContextConfig;
import org.apache.camel.CamelContext;
import org.apache.camel.EndpointInject;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.kafka.KafkaComponent;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

/**
 * Created by tmichels on 9/3/15.
 */

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = {KafkaCamelComponent.class,
                                KafkaCamelRoute.class,
                                CamelContextConfig.class})
public class KafkaCamelIntegrationTest {

    @Autowired
    CamelContext camelContext;

    @Autowired
    KafkaComponent kafkaComponent;

    @Autowired
    @Qualifier("KafkaRouteProducer")
    RouteBuilder kafkaRouteProducer;

    @Autowired
    @Qualifier("KafkaRouteConsumer")
    RouteBuilder kafkaRouteConsumer;

    @EndpointInject(uri = "direct:kafkaRoute")
    ProducerTemplate kafkaProducer;
    
    @Before
    public void before() throws Exception {
        camelContext.addRoutes(kafkaRouteProducer);
        camelContext.addRoutes(kafkaRouteConsumer);
    }

    @Test
    public void testKafkaRoute(){
        kafkaProducer.sendBody("direct:KafkaRouteProducer", "testKafkaMessage");
        try {
            camelContext.start();
            camelContext.stop();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Google Adwords Java Integration

Step 1: Create Google AdwordSession configuration

package com.integration.adwords.config;

import com.google.api.ads.adwords.axis.factory.AdWordsServices;
import com.google.api.ads.adwords.lib.client.AdWordsSession;
import com.google.api.ads.adwords.lib.client.reporting.ReportingConfiguration;
import com.google.api.ads.common.lib.auth.OfflineCredentials;
import com.google.api.ads.common.lib.exception.OAuthException;
import com.google.api.ads.common.lib.exception.ValidationException;
import com.google.api.client.auth.oauth2.Credential;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by tmichels on 7/31/15.
 */

@Configuration
public class GoogleAdwordsConfig {

    @Value("${GOOGLE_ADWORD_CLIENTID}")
    private String googleAdWordClientId;

    @Value("${GOOGLE_ADWORD_SECRET}")
    private String googleAdwordSecet;

    @Value("${GOOGLE_ADWORDS_DEV_TOKEN}")
    private String googleAdwordDevToken;

    @Value("${GOOGLE_ADWORDS_REFRESH_TOKEN}")
    private String googleAdwordRefreshToken;

    @Value("${GOOGLE_ADWORDS_USER_AGENT}")
    private String googleAdwordUserAgent;

    @Value("${GOOGLE_ADWORDS_CLIENT_CUSTOMER_ID}")
    private String googleAdwordClientCustomerId;

    @Bean
    public Credential credential() throws ValidationException, OAuthException {
        Credential credential = new OfflineCredentials.Builder()
                                .forApi(OfflineCredentials.Api.ADWORDS)
                                .withClientSecrets(googleAdWordClientId, googleAdwordSecet)
                                .withRefreshToken(googleAdwordRefreshToken)
                                .build()
                                .generateCredential();
        return credential;
    }

    @Bean
    public AdWordsSession adWordsSession(Credential credential) throws ValidationException {
        AdWordsSession adWordsSession = new AdWordsSession.Builder()
                                        .withDeveloperToken(googleAdwordDevToken)
                                        .withOAuth2Credential(credential)
                                        .withClientCustomerId(googleAdwordClientCustomerId)
                                        .withUserAgent(googleAdwordUserAgent)


                .build();
        return adWordsSession;
    }
}

2. Create AdWord Service to export KeyWords, Campaigns, AdGroups and Ads.

package com.integration.adwords.service;

import com.google.api.ads.adwords.axis.factory.AdWordsServices;
import com.google.api.ads.adwords.axis.utils.v201506.SelectorBuilder;
import com.google.api.ads.adwords.axis.v201506.cm.*;
import com.google.api.ads.adwords.lib.client.AdWordsSession;
import com.google.api.ads.adwords.lib.selectorfields.v201502.cm.CampaignField;
import com.google.api.ads.adwords.lib.selectorfields.v201506.cm.AdGroupAdField;
import com.google.api.ads.adwords.lib.selectorfields.v201506.cm.AdGroupCriterionField;
import com.google.api.ads.adwords.lib.selectorfields.v201506.cm.AdGroupField;
import com.integration.file.FileConverter;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.List;

/**
 * Created by tmichels on 7/31/15.
 */

@Service
public class GoogleAdwordService {

    private static final Logger logger = org.slf4j.LoggerFactory.getLogger(GoogleAdwordService.class);

    private static final int PAGE_SIZE = 100;

    private static final String CAMPAIGNFILENAME="adwords/google/googleAdwordCampaign.csv";
    private static final String KEYWORDFILENAME="adwords/google/googleAdwordKeywords.csv";
    private static final String ADGROUPFILENAME="adwords/google/googleAdwordAdGroup.csv";
    private static final String ADSFILENAME="adwords/google/googleAdwordAds.csv";

    @Autowired
    AdWordsSession adWordsSession;

    @Autowired
    FileConverter fileConverter;

    public void getCampaigns() throws RemoteException {
        List<String> campaignColumns = new ArrayList<>();
        List<String> campaignData = new ArrayList<>();

        // Get the CampaignService.
        CampaignServiceInterface campaignService =
                new AdWordsServices().get(adWordsSession, CampaignServiceInterface.class);

        int offset = 0;

        // Create selector.
        SelectorBuilder builder = new SelectorBuilder();
        Selector selector = builder
                .fields(CampaignField.Id,
                        CampaignField.Name,
                        CampaignField.AdServingOptimizationStatus,
                        CampaignField.AdvertisingChannelSubType,
                        CampaignField.AdvertisingChannelType,
                        CampaignField.BiddingStrategyId,
                        CampaignField.BiddingStrategyName,
                        CampaignField.BiddingStrategyType,
                        CampaignField.BidType,
                        CampaignField.BudgetId,
                        CampaignField.BudgetName,
                        CampaignField.BudgetReferenceCount,
                        CampaignField.BudgetStatus,
                        CampaignField.DeliveryMethod,
                        CampaignField.Eligible,
                        CampaignField.EndDate,
                        CampaignField.FrequencyCapMaxImpressions,
                        CampaignField.Labels,
                        CampaignField.StartDate,
                        CampaignField.TrackingUrlTemplate,
                        CampaignField.UrlCustomParameters)
                .orderAscBy(CampaignField.Id)
                .offset(offset)
                .limit(PAGE_SIZE)
                .build();

        campaignColumns.add(CampaignField.Id.toString());
        campaignColumns.add(CampaignField.Name.toString());
        campaignColumns.add(CampaignField.AdServingOptimizationStatus.toString());
        campaignColumns.add(CampaignField.AdvertisingChannelSubType.toString());
        campaignColumns.add(CampaignField.AdvertisingChannelType.toString());
        campaignColumns.add(CampaignField.BiddingStrategyId.toString());
        campaignColumns.add(CampaignField.BiddingStrategyName.toString());
        campaignColumns.add(CampaignField.BiddingStrategyType.toString());
        campaignColumns.add(CampaignField.BidType.toString());
        campaignColumns.add(CampaignField.BudgetId.toString());
        campaignColumns.add(CampaignField.BudgetName.toString());
        campaignColumns.add(CampaignField.BudgetReferenceCount.toString());
        campaignColumns.add(CampaignField.BudgetStatus.toString());
        campaignColumns.add(CampaignField.DeliveryMethod.toString());
        campaignColumns.add(CampaignField.Eligible.toString());
        campaignColumns.add(CampaignField.EndDate.toString());
        campaignColumns.add(CampaignField.FrequencyCapMaxImpressions.toString());
        campaignColumns.add(CampaignField.Labels.toString());
        campaignColumns.add(CampaignField.StartDate.toString());
        campaignColumns.add(CampaignField.TrackingUrlTemplate.toString());
        campaignColumns.add(CampaignField.UrlCustomParameters.toString());

        CampaignPage page = null;
        do {
            // Get all campaigns.
            page = campaignService.get(selector);

            // Display campaigns.
            if (page.getEntries() != null) {
                for (Campaign campaign : page.getEntries()) {
                    campaignData.add(campaign.getId().toString());
                    campaignData.add(campaign.getName());
                    campaignData.add(campaign.getAdServingOptimizationStatus()!=null? campaign.getAdServingOptimizationStatus().toString() : "");
                    campaignData.add(campaign.getAdvertisingChannelSubType() != null ? campaign.getAdvertisingChannelSubType().toString() : "");
                    campaignData.add(campaign.getAdvertisingChannelType().toString());
                    campaignData.add(campaign.getBiddingStrategyConfiguration().getBiddingStrategyId() != null ? campaign.getBiddingStrategyConfiguration().getBiddingStrategyId().toString() : "");
                    campaignData.add(campaign.getBiddingStrategyConfiguration().getBiddingStrategyName());
                    campaignData.add(campaign.getBiddingStrategyConfiguration().getBiddingStrategyType() != null ? campaign.getBiddingStrategyConfiguration().getBiddingStrategyType().getValue() : "");
                    campaignData.add(campaign.getBudget().getBudgetId() != null ? campaign.getBudget().getBudgetId().toString() : "");
                    campaignData.add(campaign.getBudget().getName());
                    campaignData.add(campaign.getBudget().getReferenceCount() != null ? campaign.getBudget().getReferenceCount().toString() : "");
                    campaignData.add(campaign.getBudget().getStatus() != null ? campaign.getBudget().getStatus().getValue() : "");
                    campaignData.add(campaign.getBudget().getDeliveryMethod() != null ? campaign.getBudget().getDeliveryMethod().getValue() : "");
                    campaignData.add(campaign.getBudget().getIsExplicitlyShared() != null ? campaign.getBudget().getIsExplicitlyShared().toString() : "");
                    campaignData.add(campaign.getEndDate());
                    campaignData.add(campaign.getFrequencyCap()!=null && campaign.getFrequencyCap().getImpressions() != null ? campaign.getFrequencyCap().getImpressions().toString() : "");
                    campaignData.add(campaign.getLabels() != null ? campaign.getLabels().toString() : "");
                    campaignData.add(campaign.getStartDate());
                    campaignData.add(campaign.getTrackingUrlTemplate());
                    campaignData.add(campaign.getUrlCustomParameters()!=null &&  campaign.getUrlCustomParameters().getParameters() != null ? campaign.getUrlCustomParameters().getParameters().toString() : "");
                }
            } else {
                logger.info("No campaigns were found.");
            }

            offset += PAGE_SIZE;
            selector = builder.increaseOffsetBy(PAGE_SIZE).build();
        } while (offset < page.getTotalNumEntries());

        fileConverter.convertToCSV(campaignData, campaignColumns, CAMPAIGNFILENAME);
    }

    public void getKeyWords() throws RemoteException {
        List<String> campaignColumns = new ArrayList<>();
        List<String> campaignData = new ArrayList<>();

        // Get the AdGroupCriterionService.
        AdGroupCriterionServiceInterface adGroupCriterionService =
                new AdWordsServices().get(adWordsSession, AdGroupCriterionServiceInterface.class);

        int offset = 0;
        boolean morePages = true;

        // Create selector.
        SelectorBuilder builder = new SelectorBuilder();
        Selector selector = builder
                .fields(
                        AdGroupCriterionField.Id,
                        AdGroupCriterionField.AdGroupId,
                        AdGroupCriterionField.KeywordMatchType,
                        AdGroupCriterionField.KeywordText)
                .offset(offset)
                .limit(PAGE_SIZE)
                .equals(AdGroupCriterionField.CriteriaType, "KEYWORD")
                .build();

        campaignColumns.add(AdGroupCriterionField.Id.toString());
        campaignColumns.add(AdGroupCriterionField.AdGroupId.toString());
        campaignColumns.add(AdGroupCriterionField.KeywordMatchType.toString());
        campaignColumns.add(AdGroupCriterionField.KeywordText.toString());

        while (morePages) {
            AdGroupCriterionPage page = adGroupCriterionService.get(selector);

            if (page.getEntries() != null && page.getEntries().length > 0) {
                for (AdGroupCriterion adGroupCriterionResult : page.getEntries()) {
                    campaignData.add(adGroupCriterionResult.getAdGroupId().toString());
                    campaignData.add(adGroupCriterionResult.getCriterion().getId().toString());
                    campaignData.add(((Keyword) adGroupCriterionResult.getCriterion()).getText());
                    campaignData.add(((Keyword) adGroupCriterionResult.getCriterion()).getMatchType().toString());
                }
            } else {
                logger.info("No keywords were found.");
            }
            offset += PAGE_SIZE;
            selector = builder.increaseOffsetBy(PAGE_SIZE).build();
            morePages = offset < page.getTotalNumEntries();
        }

        fileConverter.convertToCSV(campaignData, campaignColumns, KEYWORDFILENAME);
    }

    public void getTextAds() throws RemoteException {
        List<String> campaignColumns = new ArrayList<>();
        List<String> campaignData = new ArrayList<>();

        AdGroupAdServiceInterface adGroupAdService =
                new AdWordsServices().get(adWordsSession, AdGroupAdServiceInterface.class);

        int offset = 0;
        boolean morePages = true;

        // Create selector.
        SelectorBuilder builder = new SelectorBuilder();
        Selector selector = builder
                .fields(AdGroupAdField.Id,
                        AdGroupAdField.AdGroupId,
                        AdGroupAdField.Status)
                .orderAscBy(AdGroupAdField.Id)
                .offset(offset)
                .limit(PAGE_SIZE)
                .in(AdGroupAdField.Status, "ENABLED", "PAUSED", "DISABLED")
                .build();

        campaignColumns.add(AdGroupAdField.Id.toString());
        campaignColumns.add(AdGroupAdField.AdGroupId.toString());
        campaignColumns.add(AdGroupAdField.Status.toString());

        while (morePages) {
            AdGroupAdPage page = adGroupAdService.get(selector);

            if (page.getEntries() != null && page.getEntries().length > 0) {
                for (AdGroupAd adGroupAd : page.getEntries()) {
                    campaignData.add(adGroupAd.getAd().getId().toString());
                    campaignData.add(adGroupAd.getAd().getAdType().toString());
                    campaignData.add(adGroupAd.getStatus().toString());
                }
            } else {
                logger.info("No text ads were found");
            }

            offset += PAGE_SIZE;
            selector = builder.increaseOffsetBy(PAGE_SIZE).build();
            morePages = offset < page.getTotalNumEntries();
        }

        fileConverter.convertToCSV(campaignData, campaignColumns, ADSFILENAME);

    }
    public void getAdGroups() throws RemoteException {
        List<String> campaignColumns = new ArrayList<>();
        List<String> campaignData = new ArrayList<>();

        AdGroupServiceInterface adGroupService = new AdWordsServices().get(adWordsSession, AdGroupServiceInterface.class);

        int offset = 0;
        boolean morePages = true;

        // Create selector.
        SelectorBuilder builder = new SelectorBuilder();
        Selector selector = builder
                .fields(AdGroupField.Id,
                        AdGroupField.Name,
                        AdGroupField.Status)
                .orderAscBy(AdGroupField.Name)
                .offset(offset)
                .limit(PAGE_SIZE)
                .build();

        campaignColumns.add(AdGroupField.Id.toString());
        campaignColumns.add(AdGroupField.Name.toString());
        campaignColumns.add(AdGroupField.Status.toString());

        while (morePages) {
            AdGroupPage page = adGroupService.get(selector);
            // Display ad groups.
            if (page.getEntries() != null) {
                for (AdGroup adGroup : page.getEntries()) {
                    campaignData.add(adGroup.getId().toString());
                    campaignData.add(adGroup.getName().toString());
                    campaignData.add(adGroup.getStatus()!=null ? adGroup.getStatus().getValue() : "");
                }
            } else {
                logger.info("No group ads were found.");
            }

            offset += PAGE_SIZE;
            selector = builder.increaseOffsetBy(PAGE_SIZE).build();
            morePages = offset < page.getTotalNumEntries();
        }
        fileConverter.convertToCSV(campaignData, campaignColumns, ADGROUPFILENAME);
    }
}

Apache Camel Salesforce Integration

1. Setup SalesforceLoginConfig
2. Setup SalesforceCamelEndpointConfig
3. Setup SalesforceCamelComponent
4. Setup SalesforceCamelRouteConfig
5. Run SalesforceCamelIntegrationTest

1. Setup SalesforceLoginConfig

package com.sforce.core.camel;

import org.apache.camel.CamelContext;
import org.apache.camel.component.salesforce.SalesforceComponent;
import org.apache.camel.component.salesforce.SalesforceEndpointConfig;
import org.apache.camel.component.salesforce.SalesforceLoginConfig;
import org.apache.camel.impl.DefaultCamelContext;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by tmichels on 4/8/15.
 */

@Configuration
public class SalesforceCamelConfig {

    @Value("${SF_USERNAME}")
    private String username;

    @Value("${SF_PASSWORD}")
    private String password;

    @Value("${SF_TOKEN}")
    private String token;

    @Value("${SF_BASE_URL}")
    private String url;

    @Value("${SF_VERSION}")
    private String version;

    @Value("${SF_CLIENT_ID}")
    private String clientId;

    @Value("${SF_CLIENT_SECRET}")
    private String clientSecret;


    @Bean
    public SalesforceLoginConfig salesforceLoginConfig(){
        SalesforceLoginConfig salesforceLoginConfig = new SalesforceLoginConfig();
        salesforceLoginConfig.setUserName(username);
        salesforceLoginConfig.setPassword(password+token);
        salesforceLoginConfig.setLoginUrl(url);
        salesforceLoginConfig.setClientId(clientId);
        salesforceLoginConfig.setClientSecret(clientSecret);
        salesforceLoginConfig.setLazyLogin(false);
        return salesforceLoginConfig;
    }
}

2. Setup SalesforceCamelEndpointConfig

package com.sforce.core.camel;

import org.apache.camel.Endpoint;
import org.apache.camel.component.salesforce.SalesforceComponent;
import org.apache.camel.component.salesforce.SalesforceEndpoint;
import org.apache.camel.component.salesforce.SalesforceEndpointConfig;
import org.apache.camel.component.salesforce.internal.OperationName;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by tmichels on 4/13/15.
 */

@Configuration
public class SalesforceCamelEndpointConfig {
    
    @Bean
    public SalesforceEndpointConfig salesforceEndpointConfig(){
        SalesforceEndpointConfig salesforceEndpointConfig = new SalesforceEndpointConfig();
        salesforceEndpointConfig.setApiVersion("33.0");
        return salesforceEndpointConfig;
    }
}

3. Setup SalesforceCamelComponent

package com.sforce.core.camel;


import org.apache.camel.component.salesforce.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;

/**
 * Created by tmichels on 4/13/15.
 */

@Configuration
@Import({SalesforceCamelEndpointConfig.class, SalesforceCamelConfig.class})
public class SalesforceCamelComponent {

    @Autowired
    SalesforceEndpointConfig salesforceEndpointConfig;

    @Autowired
    SalesforceLoginConfig salesforceLoginConfig;

    @Bean
    public SalesforceComponent salesforceComponent(){
        SalesforceComponent salesforceComponent = new SalesforceComponent();
        salesforceComponent.setConfig(salesforceEndpointConfig);
        salesforceComponent.setLoginConfig(salesforceLoginConfig);
        salesforceComponent.setPackages("com.sfore.core.camel");
        return salesforceComponent;
    }
}

4. Setup SalesforceCamelRouteConfig

package com.sforce.core.camel;

import com.sforce.core.config.LcGroovyPropertiesConfig;
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.salesforce.SalesforceComponent;
import org.apache.camel.spring.SpringCamelContext;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.*;
import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration;
import org.apache.camel.spring.javaconfig.Main;

/**
 * Created by tmichels on 4/10/15.
 */

@Configuration
@ComponentScan({"com.sforce.core.camel"})
public class SalesforceCamelRouteConfig extends SingleRouteCamelConfiguration implements InitializingBean {

    @Autowired
    SalesforceComponent salesforceCamelComponent;

    public static void main(String[] args) throws Exception {
        Main main = new Main();
        main.run(args);
    }


    @Override
    protected CamelContext createCamelContext() throws Exception {
        SpringCamelContext camelContext = new SpringCamelContext();
        camelContext.setApplicationContext(getApplicationContext());
        camelContext.addComponent("salesforce", salesforceCamelComponent);
        camelContext.addRoutes(route());
        camelContext.start();
        return camelContext;
    }


    @Override
    public void afterPropertiesSet() throws Exception {}

    @Bean
    @Override
    public RouteBuilder route() {
        return new RouteBuilder() {
            public void configure() {
                from("direct:getBasicInfo").to("salesforce:getBasicInfo").bean(BasicInfoBean.class);
            }
        };
    }

    public static class BasicInfoBean {
        public void someMethod(String body) {
            System.out.println(">>>>>>> Salesforce Basics Info: " + body);
        }
    }
}

5. Run SalesforceCamelIntegrationTest

package com.sforce.core.camel;

import org.apache.camel.CamelContext;
import org.apache.camel.EndpointInject;
import org.apache.camel.Produce;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.component.mock.MockEndpoint;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

/**
 * Created by tmichels on 7/27/15.
 */

@ContextConfiguration(classes = {SalesforceCamelRouteConfig.class})
@RunWith(SpringJUnit4ClassRunner.class)
public class SalesforceCamelIntegrationTest {

    @Autowired
    CamelContext camelContext;

    @EndpointInject(uri = "direct:getBasicInfo")
    ProducerTemplate producer;

    @Test
    public void testCamelContext() throws Exception {
        String body = "Account";
        producer.sendBody(body);
    }
}

OUTPUT:

>>>>>>> Salesforce Basics Info: {"objectDescribe":{"name":"Account","label":"Account","updateable":true,"keyPrefix":"001","custom":false,"urls":{"sobject":"/services/data/v33.0/sobjects/Account","describe":"/services/data/v33.0/sobjects/Account/describe","rowTemplate":"/services/data/v33.0/sobjects/Account/{ID}","approvalLayouts":"/services/data/v33.0/sobjects/Account/describe/approvalLayouts","quickActions":"/services/data/v33.0/sobjects/Account/quickActions","listviews":"/services/data/v33.0/sobjects/Account/listviews","layouts":"/services/data/v33.0/sobjects/Account/describe/layouts","compactLayouts":"/services/data/v33.0/sobjects/Account/describe/compactLayouts"},"searchable":true,"labelPlural":"Accounts","layoutable":true,"activateable":false,"createable":true,"deprecatedAndHidden":false,"deletable":true,"customSetting":false,"feedEnabled":true,"mergeable":true,"queryable":true,"replicateable":true,"retrieveable":true,"undeletable":true,"triggerable":true},"recentItems":[{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhUzlIAF"},"Name":"Pope John XXIII Regional High School","Id":"001W000000JhUzlIAF"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhZFQIA3"},"Name":"A PLAZA DRIVING SCHOOL","Id":"001W000000JhZFQIA3"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JgxxXIAR"},"Name":"Advanced Reproductive Care, Inc. (ARC)","Id":"001W000000JgxxXIAR"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JrwqvIAB"},"Name":"name","Id":"001W000000JrwqvIAB"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhXSnIAN"},"Name":"Stella Niagara Education Park","Id":"001W000000JhXSnIAN"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000Jr1WOIAZ"},"Name":"A J DIANA SONS INC","Id":"001W000000Jr1WOIAZ"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000Jr1TIIAZ"},"Name":"A J DIANA SONS COMPANY","Id":"001W000000Jr1TIIAZ"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000Jr1WNIAZ"},"Name":"A J DIANA SONS INC","Id":"001W000000Jr1WNIAZ"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JqzmSIAR"},"Name":"A J DIANA SONS INC","Id":"001W000000JqzmSIAR"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000Jq8TeIAJ"},"Name":"A J DIANA SONS COMPANY","Id":"001W000000Jq8TeIAJ"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhZLTIA3"},"Name":"name","Id":"001W000000JhZLTIA3"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000Jq8LVIAZ"},"Name":"A PLAZA DRIVING SCHOOL","Id":"001W000000Jq8LVIAZ"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhZBTIA3"},"Name":"A J DIANA SONS INC","Id":"001W000000JhZBTIA3"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JgxwjIAB"},"Name":"[First]Name [Last]Name","Id":"001W000000JgxwjIAB"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhUlVIAV"},"Name":"Central Texas Christian School","Id":"001W000000JhUlVIAV"},{"attributes":{"type":"Account","url":"/services/data/v33.0/sobjects/Account/001W000000JhKbtIAF"},"Name":"Sylvan Learning Center - 112","Id":"001W000000JhKbtIAF"}]}

Salesforce Integration JUnit Testing Using Spring

Spring Configuration to test integration to Salesforce using PartnerConnection, EnterpriseConnection and BulkConnection. Create a properties file called salesforcesync.properties with all the login details

salesforcesync.properties

salesforce.sandbox.username=***your username ***
salesforce.sandbox.password=***your password + security token ***
salesforce.sandbox.url=https://test.salesforce.com/services/Soap/u/29.0
salesforce.sandbox.enterprise.url=https://test.salesforce.com/services/Soap/c/29.0/
salesforce.sandbox.version=29.0

salesforce.production.username=***your username ***
salesforce.production.password=***your password + security token ***
salesforce.production.url=https://login.salesforce.com/services/Soap/u/29.0
salesforce.production.enterprise.url=https://login.salesforce.com/services/Soap/c/29.0
salesforce.production.version=29.0

Salesforce Authentication Spring Configuration

package com.sforce.authentication;

import javax.inject.Inject;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.core.env.Environment;

import com.sforce.async.AsyncApiException;
import com.sforce.async.BulkConnection;

import com.sforce.soap.partner.Connector;
import com.sforce.soap.partner.PartnerConnection;
import com.sforce.ws.ConnectionException;
import com.sforce.ws.ConnectorConfig;

/**
 * @author tmichels
 */
@PropertySource("classpath:/salesforcesync.properties")
@Configuration
public class SalesforceAuthenticationConfigImpl implements SalesforceAuthenticationConfig {

	@Inject
    private Environment environment;
		    
	private PartnerConnection partnerConnection = null;
	private BulkConnection bulkConnection = null;
	
    @Bean(name="loginToProductionSalesforce")
	public PartnerConnection loginToProductionSalesforce() {
    	try {
    		ConnectorConfig config = new ConnectorConfig();
    		config.setUsername(environment.getProperty("salesforce.production.username"));
    		config.setPassword(environment.getProperty("salesforce.production.password"));
			partnerConnection = Connector.newConnection(config);
		} catch (ConnectionException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return partnerConnection;
	}

	@Bean(name="loginToSandboxSalesforce")
	public PartnerConnection loginToSandboxSalesforce() {
		try {
			ConnectorConfig config = new ConnectorConfig();
			config.setUsername(environment.getProperty("salesforce.sandbox.username"));
			config.setPassword(environment.getProperty("salesforce.sandbox.password"));
			config.setAuthEndpoint(environment.getProperty("salesforce.sandbox.url"));
			partnerConnection = Connector.newConnection(config);
		} catch (ConnectionException ce) {
			ce.printStackTrace();
		}
		return partnerConnection;
	}

	
	@Bean(name="bulkLoginToProductionSalesforce")
	public BulkConnection bulkLoginToProductionSalesforce() {
		try {
			
			ConnectorConfig partnerConfig = new ConnectorConfig();
			partnerConfig.setUsername(environment.getProperty("salesforce.production.username"));
			partnerConfig.setPassword(environment.getProperty("salesforce.production.password"));
			partnerConfig.setAuthEndpoint(environment.getProperty("salesforce.production.url"));
			partnerConnection = Connector.newConnection(partnerConfig);
			
		    ConnectorConfig config = new ConnectorConfig();
		    config.setSessionId(partnerConfig.getSessionId());
		    String soapEndpoint = partnerConfig.getServiceEndpoint();
		    String restEndpoint = soapEndpoint.substring(0, soapEndpoint.indexOf("Soap/"))
		        + "async/" + environment.getProperty("salesforce.sandbox.version");
		    config.setRestEndpoint(restEndpoint);
		    // This should only be false when doing debugging.
		    config.setCompression(true);
		    // Set this to true to see HTTP requests and responses on stdout
		    config.setTraceMessage(false);
		    bulkConnection = new BulkConnection(config);
		} catch (AsyncApiException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (ConnectionException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return bulkConnection;
	}

	@Bean(name="bulkLoginToSandboxSalesforce")
	public BulkConnection bulkLoginToSandboxSalesforce() {
		try {
			ConnectorConfig partnerConfig = new ConnectorConfig();
			partnerConfig.setUsername(environment.getProperty("salesforce.sandbox.username"));
			partnerConfig.setPassword(environment.getProperty("salesforce.sandbox.password"));
			partnerConfig.setAuthEndpoint(environment.getProperty("salesforce.sandbox.url"));
			partnerConnection = Connector.newConnection(partnerConfig);
			
		    ConnectorConfig bulkconfig = new ConnectorConfig();
		    bulkconfig.setSessionId(partnerConfig.getSessionId());
		    String soapEndpoint = partnerConfig.getServiceEndpoint();
		    String restEndpoint = soapEndpoint.substring(0, soapEndpoint.indexOf("Soap/"))
		        + "async/" + environment.getProperty("salesforce.sandbox.version");
		     bulkconfig.setRestEndpoint(restEndpoint);
		    // This should only be false when doing debugging.
		    bulkconfig.setCompression(true);
		    // Set this to true to see HTTP requests and responses on stdout
		    bulkconfig.setTraceMessage(false);
		    bulkConnection = new BulkConnection(bulkconfig);
		} catch (AsyncApiException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (ConnectionException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return bulkConnection;
	}

	@Bean(name="loginToSalesforceSandboxEnterprise")
	public EnterpriseConnection loginToSandboxSalesforceEnterprise() {
		try {
			ConnectorConfig config = new ConnectorConfig();
			config.setUsername(environment.getProperty("salesforce.sandbox.username"));
			config.setPassword(environment.getProperty("salesforce.sandbox.password"));
			config.setAuthEndpoint(environment.getProperty("salesforce.sandbox.enterprise.url"));
			config.setValidateSchema(true);
			config.setCompression(false);
			config.setConnectionTimeout(360000);
			config.setTraceMessage(true);
			config.setPrettyPrintXml(true);
			enterpriseConnection = new EnterpriseConnection(config);
		} catch (ConnectionException ce) {
			ce.printStackTrace();
		}
		return enterpriseConnection;
	}

	@Bean(name="loginToSalesforceProductionEnterprise")
	public EnterpriseConnection loginToProductionSalesforceEnterprise() {
		try {
			ConnectorConfig config = new ConnectorConfig();
			config.setUsername(environment.getProperty("salesforce.production.username"));
			config.setPassword(environment.getProperty("salesforce.production.password"));
			config.setAuthEndpoint(environment.getProperty("salesforce.production.url"));
			config.setValidateSchema(true);
			config.setCompression(false);
			config.setConnectionTimeout(360000);
			config.setTraceMessage(true);
			config.setPrettyPrintXml(true);
		
			enterpriseConnection = new EnterpriseConnection(config);
		} catch (ConnectionException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return enterpriseConnection;
	}
}

Junit test class to assert login was successful

package com.sforce.authentication;

import static org.junit.Assert.*;

import javax.inject.Inject;
import javax.inject.Named;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.support.AnnotationConfigContextLoader;

import com.sforce.async.BulkConnection;
import com.sforce.authentication.SalesforceAuthenticationConfig;
import com.sforce.authentication.SalesforceAuthenticationConfigImpl;

import com.sforce.soap.partner.PartnerConnection;
import com.sforce.ws.ConnectionException;

/**
 * @author tmichels
 */
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(loader=AnnotationConfigContextLoader.class, classes={SalesforceAuthenticationConfigImpl.class})
public class SalesforceAuthenticationServiceTest {

	@Inject @Named("loginToProductionSalesforce") PartnerConnection loginToProductionPartner;
	@Inject @Named("loginToSandboxSalesforce") PartnerConnection loginToSandboxPartner;
	@Inject @Named("bulkLoginToProductionSalesforce") BulkConnection loginToProductionBulkConnection;
	@Inject @Named("bulkLoginToSandboxSalesforce") BulkConnection loginToSandboxBulkConnection;
	@Inject @Named("loginToSalesforceSandboxEnterprise") EnterpriseConnection loginToSalesforceEnterpriseConnection;

	@Test
	public void testLoginToProductionSalesforce() {
		assertNotNull(loginToProductionPartner.getConfig().getSessionId());
		try {
			loginToProductionPartner.logout();
		} catch (ConnectionException e) {
			e.printStackTrace();
		}
	}
	
	@Test
	public void testLoginToSandboxSalesforce(){
		assertNotNull(loginToSandboxPartner.getConfig().getSessionId());
		try {
			loginToSandboxPartner.logout();
		} catch (ConnectionException e) {
			e.printStackTrace();
		}
	}
	
	@Test
	public void testBulkLoginToProductionSalesforce(){
		assertNotNull(loginToProductionBulkConnection.getConfig().getSessionId());
	}
	
	@Test
	public void testBulkLoginToSandboxSalesforce(){
		assertNotNull(loginToSandboxBulkConnection.getConfig().getSessionId());
	}
	
	@Test
	public void testLoginToSalesforceEnterprise(){
		assertNotNull(loginToSalesforceEnterpriseConnection.getConfig().getSessionId());
	}
}

Java Code: Apex Bulk Loading MySQL

Creating a bulk sync between MySQL and Salesforce using the Salesforce Bulk API. I created a Bulk Loader that connects to MySQL exports data to CSV and imports it into Salesforce. In this example I am syncing the Lead object.

package com.thysmichels;

import java.io.*;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.*;

import au.com.bytecode.opencsv.CSVWriter;

import com.sforce.async.*;
import com.sforce.soap.partner.PartnerConnection;
import com.sforce.ws.ConnectionException;
import com.sforce.ws.ConnectorConfig;

public class BulkLoader {

// Salesforce.com credentials
  private String userName = "yourusername";
  private String password = "yourpasswordandsecuritytoken";
  // the sObject being uploaded
  private String sObject = "Lead";
  // the CSV file being uploaded - either manually or from MySQL resultset
  // ie /Users/Me/Desktop/BulkAPI/myfile.csv
  private String csvFileName = "/Users/tmichels/Documents/lead.csv";
  // MySQL connection URL
  String mySqlUrl = "jdbc:mysql://localhost/RLSFDCArchive?user=admin&password=admin";
  // the query that returns results from MySQL
  String mySqlQuery = "SELECT LASTNAME, COMPANY, STATUS, EMAIL FROM RLSFDCArchive.LEAD limit 10";

  private BufferedReader console = null;

  public static void main(String[] args) throws AsyncApiException, ConnectionException, IOException 
  {
    BulkLoader example = new BulkLoader();
    example.run();
  }

  public void run() throws AsyncApiException, ConnectionException, IOException
  {
          if (createCSVFromMySQL()) 
          {
            System.out.println("Submitting CSV file to Salesforce.com");
            runJob(sObject, userName, password, csvFileName);
          }
}
  private boolean createCSVFromMySQL() {

    System.out.println("Fetching records from MySQL");

    Connection conn = null;
    ResultSet rs = null;
    boolean success = false;

    try {
      Class.forName("com.mysql.jdbc.Driver").newInstance();
      conn = DriverManager.getConnection(mySqlUrl);

      Statement s = conn.createStatement ();
      s.executeQuery(mySqlQuery);
      rs = s.getResultSet();

      // dump the contents to the console
      //System.out.println(rs.getMetaData().getColumnName(1));

    /* while (rs.next ()){
        String LnVal = rs.getString ("LASTNAME");
        String CompanyVal = rs.getString ("COMPANY");
        System.out.println ("LastName = "+LnVal+", Company = "+CompanyVal);           
      }   */

      // write the result set to the CSV file
      if (rs != null) {
        CSVWriter writer = new CSVWriter(new FileWriter(csvFileName), ',');
        writer.writeAll(rs, true);
        writer.close();
        System.out.println("Successfully fetched records from MySQL");
        success = true;
      }

    } catch (Exception e) {
      System.err.println("Cannot connect to database server");
      success = false;
    } finally {
      if (rs != null) {
        try {
          rs.close();
          System.out.println("Resultset terminated");
        } catch (Exception e1) { /* ignore close errors */
        }
      }
      if (conn != null) {
        try {
          conn.close();
          System.out.println("Database connection terminated");
        } catch (Exception e2) { /* ignore close errors */
        }
      }

    }
    return success;

  }

  public void runJob(String sobjectType, String userName, String password,
      String sampleFileName) throws AsyncApiException, ConnectionException,
      IOException 
{
  BulkConnection connection = getBulkConnection(userName, password);
    JobInfo job = createJob(sobjectType, connection);
    List<BatchInfo> batchInfoList = createBatchesFromCSVFile(connection, job, sampleFileName);
    closeJob(connection, job.getId());
    awaitCompletion(connection, job, batchInfoList);
    checkResults(connection, job, batchInfoList); 
     }

  private void awaitCompletion(BulkConnection connection, JobInfo job,
      List<BatchInfo> batchInfoList) throws AsyncApiException {
    long sleepTime = 0L;
    Set<String> incomplete = new HashSet<String>();
    for (BatchInfo bi : batchInfoList) {
      incomplete.add(bi.getId());
    }
    while (!incomplete.isEmpty()) {
      try {
        Thread.sleep(sleepTime);
      } catch (InterruptedException e) {
      }
      System.out.println("Awaiting results..." + incomplete.size());
      sleepTime = 10000L;
      BatchInfo[] statusList = connection.getBatchInfoList(job.getId())
          .getBatchInfo();
      for (BatchInfo b : statusList) {
        if (b.getState() == BatchStateEnum.Completed
            || b.getState() == BatchStateEnum.Failed) {
          if (incomplete.remove(b.getId())) {
            System.out.println("BATCH STATUS:\n" + b);
          }
        }
      }
    }
  }

  private void checkResults(BulkConnection connection, JobInfo job,
      List<BatchInfo> batchInfoList) throws AsyncApiException, IOException {
    // batchInfoList was populated when batches were created and submitted
    for (BatchInfo b : batchInfoList) {
      CSVReader rdr = new CSVReader(connection.getBatchResultStream(
          job.getId(), b.getId()));
      List<String> resultHeader = rdr.nextRecord();
      int resultCols = resultHeader.size();

      List<String> row;
      while ((row = rdr.nextRecord()) != null) {
        Map<String, String> resultInfo = new HashMap<String, String>();
        for (int i = 0; i < resultCols; i++) {
          resultInfo.put(resultHeader.get(i), row.get(i));
        }
        boolean success = Boolean.valueOf(resultInfo.get("Success"));
        boolean created = Boolean.valueOf(resultInfo.get("Created"));
        String id = resultInfo.get("Id");
        String error = resultInfo.get("Error");
        if (success && created) {
          System.out.println("Created row with id " + id);
        } else if (!success) {
          System.out.println("Failed with error: " + error);
        }
      }
    }
  }

  private void closeJob(BulkConnection connection, String jobId)
      throws AsyncApiException {
    JobInfo job = new JobInfo();
    job.setId(jobId);
    job.setState(JobStateEnum.Closed);
    connection.updateJob(job);
  }

  private List<BatchInfo> createBatchesFromCSVFile(BulkConnection connection,
      JobInfo jobInfo, String csvFileName) throws IOException,
      AsyncApiException {
    List<BatchInfo> batchInfos = new ArrayList<BatchInfo>();
    BufferedReader rdr = new BufferedReader(new InputStreamReader(
        new FileInputStream(csvFileName)));
    // read the CSV header row
    byte[] headerBytes = (rdr.readLine() + "\n").getBytes("UTF-8");
    int headerBytesLength = headerBytes.length;
    File tmpFile = File.createTempFile("bulkAPIInsert", ".csv");

    // Split the CSV file into multiple batches
    try {
      FileOutputStream tmpOut = new FileOutputStream(tmpFile);
      int maxBytesPerBatch = 10000000; // 10 million bytes per batch
      int maxRowsPerBatch = 10000; // 10 thousand rows per batch
      int currentBytes = 0;
      int currentLines = 0;
      String nextLine;
      while ((nextLine = rdr.readLine()) != null) {
        byte[] bytes = (nextLine + "\n").getBytes("UTF-8");
        // Create a new batch when our batch size limit is reached
        if (currentBytes + bytes.length > maxBytesPerBatch
            || currentLines > maxRowsPerBatch) {
          createBatch(tmpOut, tmpFile, batchInfos, connection, jobInfo);
          currentBytes = 0;
          currentLines = 0;
        }
        if (currentBytes == 0) {
          tmpOut = new FileOutputStream(tmpFile);
          tmpOut.write(headerBytes);
          currentBytes = headerBytesLength;
          currentLines = 1;
        }
        tmpOut.write(bytes);
        currentBytes += bytes.length;
        currentLines++;
      }
      // Finished processing all rows
      // Create a final batch for any remaining data
      if (currentLines > 1) {
        createBatch(tmpOut, tmpFile, batchInfos, connection, jobInfo);
      }
    } finally {
      tmpFile.delete();
    }
    return batchInfos;
  }

  private void createBatch(FileOutputStream tmpOut, File tmpFile,
      List<BatchInfo> batchInfos, BulkConnection connection, JobInfo jobInfo)
      throws IOException, AsyncApiException {
    tmpOut.flush();
    tmpOut.close();
    FileInputStream tmpInputStream = new FileInputStream(tmpFile);
    try {
      BatchInfo batchInfo = connection.createBatchFromStream(jobInfo,
          tmpInputStream);
      System.out.println(batchInfo);
      batchInfos.add(batchInfo);

    } finally {
      tmpInputStream.close();
    }
  }

  private BulkConnection getBulkConnection(String userName, String password)
      throws ConnectionException, AsyncApiException {
    ConnectorConfig partnerConfig = new ConnectorConfig();
    partnerConfig.setUsername(userName);
    partnerConfig.setPassword(password);
    partnerConfig.setAuthEndpoint("https://www.salesforce.com/services/Soap/u/25.0");
    // Creating the connection automatically handles login and stores
    // the session in partnerConfig
    new PartnerConnection(partnerConfig);
    // When PartnerConnection is instantiated, a login is implicitly
    // executed and, if successful,
    // a valid session is stored in the ConnectorConfig instance.
    // Use this key to initialize a RestConnection:
    ConnectorConfig config = new ConnectorConfig();
    config.setSessionId(partnerConfig.getSessionId());
    // The endpoint for the Bulk API service is the same as for the normal
    // SOAP uri until the /Soap/ part. From here it's '/async/versionNumber'
    String soapEndpoint = partnerConfig.getServiceEndpoint();
    String apiVersion = "25.0";
    String restEndpoint = soapEndpoint.substring(0, soapEndpoint
        .indexOf("Soap/"))
        + "async/" + apiVersion;
    config.setRestEndpoint(restEndpoint);
    // This should only be false when doing debugging.
    config.setCompression(true);
    // Set this to true to see HTTP requests and responses on stdout
    config.setTraceMessage(false);
    BulkConnection connection = new BulkConnection(config);
    return connection;
  }

  private JobInfo createJob(String sobjectType, BulkConnection connection)
      throws AsyncApiException 
      {
    JobInfo job = new JobInfo();
    job.setObject(sobjectType);
    job.setOperation(OperationEnum.insert);
    job.setContentType(ContentType.CSV);
    job = connection.createJob(job);
    System.out.println(job);
    return job;
      }

}

Below is the output expected:

Fetching records from MySQL
Successfully fetched records from MySQL
Resultset terminated
Database connection terminated
Submitting CSV file to Salesforce.com
[JobInfo  id='75060000000LtROAA0'
 operation='insert'
 object='Lead'
 createdById='00560000001mciHAAQ'
 createdDate='java.util.GregorianCalendar[time=1357172058000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=18,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 systemModstamp='java.util.GregorianCalendar[time=1357172058000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=18,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 state='Open'
 externalIdFieldName='null'
 concurrencyMode='Parallel'
 contentType='CSV'
 numberBatchesQueued='0'
 numberBatchesInProgress='0'
 numberBatchesCompleted='0'
 numberBatchesFailed='0'
 numberBatchesTotal='0'
 numberRecordsProcessed='0'
 numberRetries='0'
 apiVersion='25.0'
 assignmentRuleId='null'
 numberRecordsFailed='0'
 totalProcessingTime='0'
 apiActiveProcessingTime='0'
 apexProcessingTime='0'
]

[BatchInfo  id='75160000000sEGGAA2'
 jobId='75060000000LtROAA0'
 state='Queued'
 stateMessage='null'
 createdDate='java.util.GregorianCalendar[time=1357172059000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=19,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 systemModstamp='java.util.GregorianCalendar[time=1357172059000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=19,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 numberRecordsProcessed='0'
 numberRecordsFailed='0'
 totalProcessingTime='0'
 apiActiveProcessingTime='0'
 apexProcessingTime='0'
]

Awaiting results...1
Awaiting results...1
BATCH STATUS:
[BatchInfo  id='75160000000sEGGAA2'
 jobId='75060000000LtROAA0'
 state='Completed'
 stateMessage='null'
 createdDate='java.util.GregorianCalendar[time=1357172059000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=19,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 systemModstamp='java.util.GregorianCalendar[time=1357172060000,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2013,MONTH=0,WEEK_OF_YEAR=1,WEEK_OF_MONTH=1,DAY_OF_MONTH=3,DAY_OF_YEAR=3,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=1,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=14,SECOND=20,MILLISECOND=0,ZONE_OFFSET=0,DST_OFFSET=0]'
 numberRecordsProcessed='10'
 numberRecordsFailed='1'
 totalProcessingTime='753'
 apiActiveProcessingTime='648'
 apexProcessingTime='517'
]

Java Code: Salesforce Export into Excel

Export Salesforce records into Excel. In my example I read records from Excel and also from Salesforce. Reuse my code by providing your own username and password to your Salesforce org to create more advanced SFDC to Excel integrations. You will need to download or add jxl to your maven project:

http://mvnrepository.com/artifact/net.sourceforge.jexcelapi/jxl

package com.thys.michels; 
import java.io.File;
import java.io.IOException;
import java.util.Locale;

import com.sforce.soap.partner.Connector;
import com.sforce.soap.partner.DescribeSObjectResult;
import com.sforce.soap.partner.PartnerConnection;
import com.sforce.soap.partner.QueryResult;
import com.sforce.soap.partner.sobject.SObject;
import com.sforce.ws.ConnectionException;
import com.sforce.ws.ConnectorConfig;

import jxl.Cell;
import jxl.CellType;
import jxl.Sheet;
import jxl.Workbook;
import jxl.read.biff.BiffException;

import jxl.CellView;
import jxl.WorkbookSettings;
import jxl.format.UnderlineStyle;
import jxl.write.Label;
import jxl.write.WritableCellFormat;
import jxl.write.WritableFont;
import jxl.write.WritableSheet;
import jxl.write.WritableWorkbook;
import jxl.write.WriteException;
import jxl.write.biff.RowsExceededException;

public class ForcedotcomExcelToolkit {
    private WritableCellFormat times;
    private String inputFile;
    private WritableCellFormat timesBoldUnderline;

    public void setInputFile(String inputFile) {
        this.inputFile = inputFile;
    }
    public void setOutputFile(String inputFile) {
        this.inputFile = inputFile;
    }

    public String[] read() throws IOException  {
        File inputWorkbook = new File(inputFile);
        Workbook w;
        try {
            w = Workbook.getWorkbook(inputWorkbook);
// Get the first sheet
            Sheet sheet = w.getSheet(0);
// Loop over first 10 column and lines

            String arremail[] = new String[246];

            for (int j = 0; j &lt; sheet.getColumns(); j++) 
            {
                for (int i = 0; i &lt; sheet.getRows(); i++) 
                {
                    Cell cell = sheet.getCell(j, i);
                    CellType type = cell.getType();
                    if (cell.getType() == CellType.LABEL) {
                        arremail[i] = cell.getContents();
                    }
                }
            }
            return arremail;
        } 
        catch (BiffException e) 
        {
            e.printStackTrace();
            return null;
        }
    }

    public void write(String[][] arremail) throws IOException, WriteException {
        File file = new File(inputFile);
        WorkbookSettings wbSettings = new WorkbookSettings();
        wbSettings.setLocale(new Locale("en", "EN"));
        WritableWorkbook workbook = Workbook.createWorkbook(file, wbSettings);
        workbook.createSheet("Report", 0);
        WritableSheet excelSheet = workbook.getSheet(0);
        createLabel(excelSheet);
        System.out.println(arremail.length);
        for (int i = 1; i &lt; arremail.length; i++) 
        {
// First column
            if (arremail[i-1][0]!=null)
                addLabel(excelSheet, 0, i, arremail[i-1][0].toString());
// Second column
            if (arremail[i-1][1]!=null)
                addLabel(excelSheet, 1, i, arremail[i-1][1].toString());
        }
//createContent(excelSheet, arremail);
        System.out.println("Done 8");
        workbook.write();
        workbook.close();
    }
//private void createContent(WritableSheet sheet, String[][] arremail) throws WriteException,RowsExceededException {
// Write a few number

//}

    private void createLabel(WritableSheet sheet)
            throws WriteException {
// Lets create a times font
        WritableFont times10pt = new WritableFont(WritableFont.TIMES, 10);
// Define the cell format
        times = new WritableCellFormat(times10pt);
// Lets automatically wrap the cells
        times.setWrap(true);

// Create create a bold font with unterlines
        WritableFont times10ptBoldUnderline = new WritableFont(WritableFont.TIMES, 10, WritableFont.BOLD, false,
                UnderlineStyle.SINGLE);
        timesBoldUnderline = new WritableCellFormat(times10ptBoldUnderline);
// Lets automatically wrap the cells
        timesBoldUnderline.setWrap(true);

        CellView cv = new CellView();
        cv.setFormat(times);
        cv.setFormat(timesBoldUnderline);
        cv.setAutosize(true);

// Write a few headers
        addCaption(sheet, 0, 0, "Email");
        addCaption(sheet, 1, 0, "Phone");

    }
    private void addCaption(WritableSheet sheet, int column, int row, String s)
            throws RowsExceededException, WriteException {
        Label label;
        label = new Label(column, row, s, timesBoldUnderline);
        sheet.addCell(label);
    }

    private void addLabel(WritableSheet sheet, int column, int row, String s)
            throws WriteException, RowsExceededException {
        Label label;
        label = new Label(column, row, s, times);
        sheet.addCell(label);
    }

    public String[][] SFDCEmailResult(String [] stremail)
    {
//Create a new connectionconfig to your Salesforce Org
        ConnectorConfig sfconfig = new ConnectorConfig();
//Use your salesforce username = email
        sfconfig.setUsername("yourusername");
//Use your saleforce password with your security token look like: passwordjeIzBAQKkR6FBW8bw5HbVkkkk
        sfconfig.setPassword("yourpassword");
        sfconfig.setConnectionTimeout(99999);
        PartnerConnection partnercon;
        try
        {
            partnercon = Connector.newConnection(sfconfig);
            DescribeSObjectResult describeGlobalResult = partnercon.describeSObject("Contact");
//DescribeGlobalSObjectResult[] sobjectResults = describeGlobalResult.getSobjects();
            String [][] emailresult = new String[246][246];
            for (int k = 0; k &lt; stremail.length; k++)
            {
                String QueryStr = "SELECT Email, Phone FROM Contact where Email='" + stremail[k]+"'";
                QueryResult queryResults = partnercon.query(QueryStr);
                if (queryResults.getSize() &gt; 0)
                {
                    for (SObject s: queryResults.getRecords())
                    {
                        emailresult[k][0] = s.getField("Email").toString();
                        if (s.getField("Phone") != null)
                            emailresult[k][1] = s.getField("Phone").toString();
                        else
                            emailresult[k][1]="";
                    }
                }
            }
            return emailresult;
        }
        catch(ConnectionException e)
        {
            e.printStackTrace();
            return null;
        }
    }

    public static void main(String[] args) throws IOException, WriteException
    {
        ForcedotcomExcelToolkit outemailexcelinput = new ForcedotcomExcelToolkit();
        outemailexcelinput.setInputFile("/Users/tmichels/Documents/SFDCEmail.xls");
        System.out.println("Complete Reading Emails from Excel");
        ForcedotcomExcelToolkit outemailexceloutput = new ForcedotcomExcelToolkit();
        outemailexceloutput.setOutputFile("/Users/tmichels/Documents/SFDCEmailResults.xls");
        outemailexceloutput.write(outemailexceloutput.SFDCEmailResult(outemailexcelinput.read()));
        System.out.println("Complete Writing Emails");
    }

Insert row DB2 with Websphere Cast Iron

How to insert a row into DB2 using Websphere Cast Iron.

Step 1: Create an DB2 Endpoint by specifying the following information:

  • DB2 Database Name
  • DB2 Host Name
  • DB2 Port Number
  • DB2 Username
  • DB2 Password

Please make sure that the Cast Iron appliance or Virtual image has rights to insert a row onto DB2. This can be done by specifying the Privileges in the DB2 Control Centre.

Step 2: Test the connection to verify all information is correct.

Step 3: Create your integration flow

Step 4: Specify the DB2 Endpoint created in Step 1 and select the table and row you want the data to be inserted into

Step 5: Create the input and output maps for both activities

Step 6: Verify and test the flow.

This flow will update a website that is located in a cloud environment (like Amazon EC2) and updated an on-premise DB2 instance.

Cast Iron Websphere MQ integration Flow

How to integrate Websphere MQ integration flow with Cast Iron tutorial.

Before starting this tutorial make sure that you have the MQ jar files added to the -> Install Module Providers Menu

Step 1: Create a Websphere MQ endpoint by specifying the following information:

  • Websphere MQ hostname
  • Websphere MQ port
  • Queue Manager Name
  • Server Connection Channel Name
  • MQ Privilege User – Username
  • MQ Privilege User – Password

Step 2:  On Websphere MQ Explorer create the following:

  • Local Queue
  • Create Server Connection Channel

Step 3: Create your integration Flow

Step 4: Specify the the MQ Endpoint as one create in Step 1

Step 5: Create the input and output maps for the two activities.

Step 6: Verify and test the integration flow.