Happy new 2011 to all readers of this blog!

Maybe I am only an old degenerate bastard, but this cartoon makes me always laugh...

   

The adapter will not process since it has been processed earlier

<29-Dec-2010 20:27:12 o'clock WET>    <
File Adapter PVPhase1 The adapter will not process [C:\acmepoc\myFiles\in\good2.txt] since it has been processed earlier>
<29-Dec-2010 20:27:15 o'clock WET>    <
File Adapter PVPhase1 The adapter has handled the poisoned file [C:
\acmepoc\myFiles\in\good2.txt]>

still investigating..... why the File Adapter doesn't want to process the same file at 5 minutes distance? There must be some parameter somewhere, but where?

ORAMED-03601

oracle.fabric.common.FabricInvocationException: oracle.tip.mediator.infra.exception.MediatorException: ORAMED-03601:[Invalid transformer configuration]Unable to create transformer. It could be because of transformation file or configuration issues


I am struggling with XSL transformations. JDeveloper doesn't seem to react very well to manual manipulations of the file. You come back from "source" to "design" and the design doesn't reflect the connections. And you get the runtime error.

Workaround: avoid manual hacks..... :o(

tid, ecid, component_instance_id, dcid, composite_instance_id

in a SOA Suite log line, these elements appear:


[2010-12-29T16:37:30.576+00:00] [soa_server1] [NOTIFICATION] [] [oracle.soa.mediator.common.cache] [tid: weblogic.work.j2ee.J2EEWorkManager$WorkWithListener@309d309d] [userId: pierre] [ecid: 0000IomzSwp8lnc5tjw0yX1D1v7j0036DZ,0] [APP: soa-infra] [composite_name: PierreProject] [component_instance_id: EDF2C8C0136911E0BFFDEF342CAF0998] [dcid: 240449412c647dab:58beda0f:12ce60fee82:-7ffd-000000000000d44b] [component_name: Mediator1] [composite_instance_id: 4] calloutMediatorMessage.toString=oracle.tip.mediator.common.message.MediatorNormalizedMessage@41524152




What do they mean?






tid: ??

ecid: The ECID is used to track a flow messages through and between SOA Composites.

component_instance_id: the identification of the component within the composite.

dcid: ??

composite_instance_id: the identification of the (SCA) composite.


ECID, component_instance_id and composite_instance_id are used for example to show the flow of messages through the several components of you composites...

I shall answer to each ?? as I find out...

ORABPEL-11207

I was trying to write into a File using File Adapter and opaqueElement.... evidently there is a trick, I know you must transform the data in base64 binary format.... one day I will find out how to do it...

IO Failure in translator.
IO failure because the   translator failed to   to copy InputStream to OutputStream. .
Check the error stack and fix the cause of the error. Contact oracle support if error is not fixable.

 at oracle.tip.pc.services.translation.framework.XlatorHelper.copyStream(XlatorHelper.java:160)
 at oracle.tip.pc.services.translation.xlators.opaque.OpaqueTranslator.translateToNative(OpaqueTranslator.java:311)
 at oracle.tip.adapter.file.FileUtil.translate(FileUtil.java:1262)
 ... 64 more
Caused by: java.io.IOException: Error in encoded stream: needed 4 valid base64 characters but only got 3 before EOF, the 10 most recent characters were: "elloPierre"
 at com.sun.mail.util.BASE64DecoderStream.decode(BASE64DecoderStream.java:250)
 at com.sun.mail.util.BASE64DecoderStream.read(BASE64DecoderStream.java:148)
 at java.io.FilterInputStream.read(FilterInputStream.java:102)
 at oracle.tip.pc.services.translation.framework.XlatorHelper.copyStream(XlatorHelper.java:157)
 ... 66 more

Oracle Service Oriented Architecture Infrastructure Implementation Certified Expert

http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=352


Just a note to myself: if I lose my job I will dedicate some time to prepare this certification... at least I will be able to deal with "Architects" as a peer :o)

Java callout in SOA Suite

oracle.tip.mediator.common.api.IJavaCallout is the interface to implement.

Here the interface is documented.

I have been looking for the Javadoc online but could find none.

It is in C:\Oracle1\Middleware\jdeveloper\soa\modules\oracle.soa.mediator_11.1.1\mediator.jar

Voila a Java callback naked and crude:

import java.util.logging.Logger;

import oracle.tip.mediator.common.api.CalloutMediatorMessage;
import oracle.tip.mediator.metadata.CaseType;


public class MyClass implements oracle.tip.mediator.common.api.IJavaCallout {

    public void initialize(Logger logger) {
    }

    public boolean preRouting(CalloutMediatorMessage calloutMediatorMessage) {
        return false;
    }

    public boolean preRoutingRule(CaseType caseType, CalloutMediatorMessage calloutMediatorMessage) {
        return false;
    }

    public boolean postRoutingRule(CaseType caseType, CalloutMediatorMessage calloutMediatorMessage, CalloutMediatorMessage calloutMediatorMessage1, Throwable throwable) {
        return false;
    }

    public boolean postRouting(CalloutMediatorMessage calloutMediatorMessage, CalloutMediatorMessage calloutMediatorMessage1, Throwable throwable) {
        return false;
    }

    public boolean preCallbackRouting(CalloutMediatorMessage calloutMediatorMessage) {
        return false;
    }

    public boolean postCallbackRouting(CalloutMediatorMessage calloutMediatorMessage, Throwable throwable) {
        return false;
    }
}

To be noticed that the postRouting/postRoutingRule have 2 parameters, one is the request and the other the response (I guess!)

Callback has a single parameter because there is no way to remember the request to which this callback is the response.... at least not in Mediator.


Here a good Hello World tutorial on the subject.

Jdeveloper 11.1.1.3 not recognizing java package

Weird, I do
"new java package"
enter the name (com.acme.bla)
click on OK

nothing happens. The folder is not created. In the Application Navigator it doesn't show.

I create the folder manually, refresh Application Navigator, nothing, it doesn't show.

If I create manually a Java class and put into that folder, magically the package appears in Application Navigator.

XML validate that all elements have same value

I have a payload like this:

alfa>3
beta>3
gamma>3
delta>3

and I need to validate that all "number" are the same within the same element... can be 3, can be anything else.

Now, with Schematron I doubt this is possible. With XSD neither. They all seem to be NOT oriented to operate on collections, the validation scope is the single element.

We need to come up with a different strategy.

I wish there was a DSL allowing me to state:

test: assert all /lines/line/number are same within /lines
with message: 'all line number must be same'

where we establish a scope (/lines), a predicate (are same), a subject (/lines/line/number) and a selection rule (all)

Schematron!

http://www.schematron.com/


I am planning to do advanced XML validation in SOA Suite using Schematron....
a Schematron callout is natively supported by mediator, and it looks like a cool easy technology....

Here a nice tutorial.
Here an even better tutorial.

Here a XML ValidatorBuddy plugin for Altova XMLSpy.

In Oracle Soa Suite 11g Developers Guide there is a chapter on the Schematron topic.

Hahaha Schematron reminds me of Robocop - what a fantastic movie!

Groovy Callout in SOA Suite

I was musing about using Groovy to implement custom logic in SOA Suite.
I stumbled on this post covering Groovy with OSB.... interesting!

I have seen a gain of at least 300% in the mass of code produced with Groovy vs Java, so I am planning to use Groovy more often in future....

I am trying to build "complex" validation logic in Oracle Business Rules, surely you can do interesting stuff with this tool, but there is a learning curve and I am not sure how intuitive the code will be...

Reflection with Oracle DB: getting the list of all columns with a given name

SQL> select unique object_type from user_objects order by object_type;

OBJECT_TYPE
-------------------
INDEX
INDEX PARTITION
LOB
PACKAGE
PACKAGE BODY
PROCEDURE
QUEUE
SEQUENCE
SYNONYM
TABLE
TABLE PARTITION
TRIGGER
TYPE
VIEW

with USER_OBJECTS you have information only on TABLES, not on COLUMNS.

Also
Select * From User_Tables;

Select * From all_Tables;

doesn't mention COLUMNS.

To find out about COLUMNS, use ALL_TAB_COLUMNS:

SELECT OWNER, TABLE_NAME, COLUMN_NAME FROM ALL_TAB_COLUMNS WHERE COLUMN_NAME LIKE '%INSTANCE%' order by OWNER;


See http://en.wikipedia.org/wiki/Oracle_metadata for more on Oracle Metadata and useful SQL.

File Adapter metadata with SOA Suite

( the bloody blog keeps messing up with this XML.... I thought that by 2010 these technical issues would not be there any more....)

You can easily get the filename from the properties, in Mediator go to "ASSIGN VALUES" and you will find how to copy Properties to the Expression output....

You might also copy the property to the same property in Mediator and then pass it around... see here


You might get this warning:

Warning: Assigning property/constant "$in.property.jca.file.FileName" to element "$out.payload/imp1:PreactivationFile/imp1:PreactivationLines/imp1:lotInformation/imp1:fileName". Please make sure target is single leaf node, otherwise non-leaf node will contain only string value which may generate non-valid xml as per the xsd.

Am I running Java 32bit or 64bit?

in doubt, run

java -version

for 32bit, it should tell you nothing:

java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) Client VM (build 16.0-b13, mixed mode)

for 64bit, you should get an explicit message:

java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) 64-Bit Server VM (build 14.1-b02, mixed mode)

Oracle Business Rules. validating the number (count) of elements

I have to verify that the count(Lines) is a given number in a payload. Lines is represented as a List in JAXB, and it's a "element minOccurs=0 maxOccurs=unbounded" in the XSD.

In a Rule definition, there is no way to use a XPath expression on a XML Fact.
How painful.

I had to create a Function, return type in, input parameter Lines (it is a recognized type in the Facts), and use "return Lines.size()"
Then in Business Rules you can invoke the Function to compute the number of Lines.

Roundaboutish, eh?

Citrus

See this and this for a tutorial on Citrus...



I am strongly allergic to XML....

I was hoping that Citrus would provide a nice and smooth DSL to configure the tests,
but if it involves all that XML I think I will try to achieve the same with Plain Java/Groovy....


Anyway I will give it a try...

I am reading the PDF, it seems very inspiring.

I like the grooviness of Citrus.

SOA Suite File Adapter, parsing complex flat files

In reference to this post,

here is the XSD which allows you to parse a flat file made by different elements (eg first line contains the number of records, and from second line onwards come the records):

TOPLINK-4002 using Composite Sensor on SOA Suite

[TOPLINK-4002] (Oracle TopLink - 11g Release 1 (11.1.1.3.0) (Build 100323)): or acle.toplink.exceptions.DatabaseException Internal Exception: java.sql.BatchUpdateException: ORA-01461: can bind a LONG value only for insert into a LONG column

oracle.integration.platform.sensor
BEA-000000 Fabric-Sensor Unable to publish sensor data to oracle.integration.platform.sensor.publisher.db.DatabasePublisher
 Sensor Data : blablabla

I wonder if these 2 error messages are related... apparently yes....

SOA Suite and JDeveloper: wsrp-container.jar not found, adflibWorklistComponents.jar not found, com.bea.core.logging_1.6.0.0.jar not found....

Warning: Classpath entry C:\Oracle1\Middleware\jdeveloper\integration\lib\wsrp-container.jar not found.
Warning: Classpath entry C:\Oracle1\Middleware\jdeveloper\integration\lib\adflibWorklistComponents.jar not found.

Warning: Classpath entry C:\Oracle1\Middleware\modules\com.bea.core.i18n_1.6.0.0.jar not found.
Warning: Classpath entry C:\Oracle1\Middleware\modules\com.bea.core.logging_1.6.0.0.jar not found.
Warning: Classpath entry C:\Oracle1\Middleware\modules\com.oracle.ws.http_client_1.1.0.0.jar not found.
Warning: Classpath entry C:\Oracle1\Middleware\modules\org.eclipse.persistence_1.0.0.0_1-2-0.jar not found.


This is bewildering, all of a sudden I start getting these warnings.... ???

JDeveloper, SOA Suite, File Adapter, Native Format Builder: Unable to generate native format

Still investigating the issue.... how frustrating....wasted one hour on this....

I was unimpressed by the MFL in Eclipse/OSB spitting NullPointerException when the input data contains "invalid" characters, but now NFB is also putting my patience at test....

It turned out that, when you let the wizard decide the field names and types, it leaves the last one undefined - in case it was empty in the sample file. If you manually set the type to string. the wizard is able to complete successfully. What a great error handling!

Keep all your XSDs in a single place

I want to avoid copying/pasting XSDs all over the places (every single OSB project, every single SOA Suite Project....)

This article talks about this issue.

I am experimenting with Oracle Service Registry (OSR), it was a bit of a fight to install it, but finally I chose "Jetty HTTP Server" and it works....

My dream would be to register all XSDs and WSDLs in OSR and import them into OSB using the "import from UDDI" feature....

Groovy for Dummy Java Programmers (DJP) like me

http://www.infoq.com/presentations/Transforming-to-Groovy

this presentation by Venkat Subramaniam is quite cool.

CommonJ timer with OSB, SOA Suite, SOA Scheduler

an excellent document on SOA Scheduler here:

http://www.oracle.com/technetwork/middleware/soasuite/learnmore/soascheduler-186798.pdf


I have tested it and it works beautifully! You can specify the cron parameters in a single place. Of course it can be improved but it's a good starting point.
Here the code (I hope Robert Baumgartner doesn't mind).
I feel it would be better to have the Scheduler post an EDN event using
oracle.integration.platform.blocks.event.SendEvent and wake up a Mediator instance,
rather than synchronously invoke the WS to trigger the BPEL process.

This SendEvent class is in C:\Oracle1\Middleware\jdeveloper\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar
Unfortunately it looks like the only way to use it is by invoking its main(...) procedure after having set the System properties with: dbconn,dbuser, dbpass, eventName, eventBody,event parameters. Internally it uses a SAQRemoteBusinessEventConnectionFactory and a BusinessEventBuilder. All this make it a bit complicated to generate events programmatically.


here is CommonJ doc:

http://download.oracle.com/docs/cd/E12840_01/wls/docs103/commonj/commonj.html with coverage of "cluster-wide" timers.


Here the excellent tutorial by James Bayer:
http://blogs.oracle.com/jamesbayer/2007/11/weblogic_scheduling_a_polling.html

and more here:
http://blogs.oracle.com/jamesbayer/2009/04/a_simple_job_scheduler_example.html


and a very nice tutorial on CommonJ TimerManager here
http://middlewaremagic.com/weblogic/?p=987

here the code


Other alternatives:

EJB 3.0 Timer:
http://www.javabeat.net/articles/3-ejb-30-timer-services-an-overview-1.html

more specifically in a cluster:
http://shaoxiongyang.blogspot.com/2010/10/how-to-use-ejb-3-timer-in-weblogic-10.html


I really feel a pain comparing the SOA Suite support for scheduling with the one offered by Apache Camel:
http://camel.apache.org/quartz.html



(17-Jan-2011) I discover there is a PICK activity in BPEL allowing you to schedule a BPEL process at given interval. This is most likely the simplest solution:

http://download.oracle.com/docs/cd/B31017_01/integrate.1013/b28981/events.htm#CHDECBGD

Why should I use a BPEL engine?

I found this wonderful presentation:

http://www.ibm.com/developerworks/websphere/library/techarticles/0803_fasbinder2/0803_fasbinder2.html

basically, use OSB when you have to do this:

validation
protocol mediation
transformations
messages enrichment
simple service composition with no compensation
static ou dynamic routing
load balancing and fault tolerance
error handling
monitoring performance and SLA


use a BPEL Engine for:
complex long running processes
service composition with compensation and complex orchestration

Remote monitoring of a Java application with JConsole

see http://download.oracle.com/javase/1.5.0/docs/guide/management/agent.html#remote

and better still here http://www.performanceengineer.com/blog/monitoring-weblogic-using-jmx/

set these properties:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=8888
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false



CAVEAT: on the same machine you could have multiple WLS.
You cannot assign the same jmx port to all of them.
You should prepare an individual script for each managed server,

where you set
export JAVA_OPTIONS="-Dcom.sun.management.jmxremote.port=8888"

and of course use a different port for each WLS.

Something like this:

startOSB.sh:

export JAVA_OPTIONS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
nohup ./startManagedWebLogic.sh osb_server1 http://myserver:7001 2>&1 > . ../servers/osb_server1/logs/osb_server1.out &



startSOA.sh:

export JAVA_OPTIONS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8889 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"

nohup ./startManagedWebLogic.sh soa_server1 http://myserver:7001 2>&1 > ../servers/soa_server1/logs/soa_server1.out &


You will need to enable these settings when you want to connect VisualVM with JMX, see here http://visualvm.java.net/jmx_connections.html

SOA book for the winter....

http://www.amazon.com/Enterprise-Service-Bus-David-Chappell/dp/0596006756

I read good things about this book.... I am not planning to finish Thomas Erl book, waaaaay too abstract for someone implementation and delivery oriented like me.

OSB and SOA Suite VISIO Stencils

This one is REALLY cool, it contains very specific component icons, although I wish it contained more JEE specific terminology (i.e. a JMS queue, for which I can only find a Channel icon) :

http://www.soapatterns.com/SOA_Visio_Stencil.zip

This one is very abstract, just like the Thomas Erl book, anyway a few components can be really useful:

http://www.eaipatterns.com/download/EIP_Visio_stencil.zip

nohup, redirecting nohup.out to a file

nohup ./startManagedWebLogic.sh osb_server1 2>&1 > ./osb_server1.out &

easy does it

startManagedWebLogic asking username and password

you have many options to get rid of the annoyance,
probably the quickest is to put a file boot.properties
with
username=weblogic
password=weblogic1

in $DOMAIN_DIR$/servers/myserver/security
directory


at the next boot it should be encrypted by WLS

Bottom-up, top-down, meet-in-the-middle...

nice article on WSDL design - it makes sense and it's practical...

http://www.ibm.com/developerworks/architecture/library/ar-servdsgn1/

Sequence Diagrams for The People

this tool is really cool, it creates accurate SDs in a matter of seconds...

http://sdedit.sourceforge.net/example/index.html

Great job Herr Markus Strauch !

Here is a sample script:

crm:CRM[a]
osb:OSB[a]
hdt:HDT[a]


*1 crm
CRM requests creation
of a Company/Person account
*1
crm:osb.H_REQ_CLIENT()


[c:alt nodeLevel IS NOT defined]
osb:hdt.ciaoScemo()
--[else]
osb:nothing to do
[/c]


osb:hdt.hello()

and here is the SD originated:

Sweet and simple.

JDeveloper and HttpClient to test a OSB "any XML" service

I want to code some Unit Tests for a Pure XML service.
SOAPUI doesn't seem to have any support for such services (maybe?).

So I will code something in Java. Actually I might do it in Groovy.

HttpClient is deployed 3 times (!!!) with SOASuite and JDev:

C:\Oracle1\Middleware\jdeveloper\communications\modules\oracle.sdp.client_11.1.1\commons-httpclient-3.1.jar

C:\Oracle1\Middleware\jdeveloper\ide\lib\commons-httpclient-3.1.jar

C:\Oracle1\Middleware\Oracle_SOA1\soa\modules\commons-httpclient-3.1.jar

In reality httpclient 3 is a legacy code and one is encouraged to upgrade to 4
http://hc.apache.org/downloads.cgi

(download both core and httpclient jars)
here is the link with the dependencies.


Here a quick tutorial.

Bear in mind that even if you post a XML to a "Any XML" service, OSB will still wrap it into a soapenv:Body element....

so if you want to grab the payload you should get the XPath "./*" of variable $body...
using the XPath "." will return you an object of type Body.


Here is the harness:

DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost request = new HttpPost("http://localhost:7001/CreateOrder/PS/CreateOrderPS");
HttpEntity httpEntity = new FileEntity(new File("C:\\JDeveloper\\mywork\\ACME_POC\\CreateOrderTest\\resources\\createorderpayload.xml"), "text/html");
request.setEntity(httpEntity);
HttpResponse response = httpclient.execute(request);
System.out.println(response.getStatusLine().getStatusCode());

and of course the status should be 200
If it's 500 you might have a fault (e.g. validation error)


Here for my reference are the available sbresource queries:

* http://host:port/sbresource?WSDL/project/...wsdlname
* http://host:port/sbresource?POLICY/project/...policyname
* http://host:port/sbresource?MFL/project/...mflname
* http://host:port/sbresource?SCHEMA/project/...schemaname
* http://host:port/sbresource?PROXY/project/...proxyname

SOA Suite - logging in BPEL

I was wondering how to log debug information in a BPEL process,
and here I found an excellent article on using a Java embedded action.... great stuff, Brazilian friend, bem feito!

Just code this into a Java Embedded activity:

Object countLines = getVariableData("countLines");   
  
System.out.println("IN LOOP countLines=" + countLines);

/* writes to the audit trail - will then be visible in BPEL console at runtime*/
addAuditTrailEntry("countLines is: " + countLines);

This article seems to be interesting on how to use log4j from a BPEL process

See also here for more on logging XML in BPEL

Non unique body parts!

If you get this message, you better fix it, don't rely on SOAPAction for dispatching. Every operation should have a unique body structure, carrying as root element the operation name.

Most likely, the mistake has been:
declaring multiple wsdl:message with the same "part" structure.
See here for a template you can use to write your WSDL.

I know, WSDL are a pain in the neck, especially the 1.1 version.

Elements of XSD reusability

If you define a type in a XSD:


myxsd1.xds:

you can reuse this type definition in another XSD:


myxsd2.xds:

The key is the "xs:import" statement with the association to the same namespace as in the targetNamespace used for the myxsd1.xds.

What I really don't like in XSD is the habit of defining a type as an element:

xs:element ... xs:complexType

this is plain stupid since you cannot reuse this type definition.


I mean, once you have defined an element A as of complexType sequence bla,
you cannot define an element B saying "make it just like element A".

The only way to reuse it is by including an object of type A inside the object B using the "ref" construct:

xs:element name="PVElement3" ... xs:complexType ... xs:sequence... xs:element ref="PVElement2"

This is like going back a century in the art of programming.

I keep wondering what the Fathers of XML had smoked when they got together to define the Foundation of the Second Biggest Blunder of the Century - the First being HTML.

Release It !

Luciano has recommended me this book, he says it's full of insight on how to build testable applications.

I have started reading it and I am enjoying every line I read.

This is a great quote from the book:

Your early decisions make the biggest impact on the eventual shape of
your system. The earliest decisions you make can be the hardest ones
to reverse later. These early decisions about the system boundary and
decomposition into subsystems get crystallized into the team structure,
funding allocation, program management structure, and even timesheet
codes. Team assignments are the first draft of the architecture.
It’s a terrible irony that these very early
decisions are also the least informed. This is when your team is most
ignorant of the eventual structure of the software in the beginning, yet
that is when some of the most irrevocable decisions must be made.

Indirectly referenced variables in BPEL

or "Dynamically specified XPaths in BPEL"

Let's assume I have a String variable "counter",
and I want to do a lookup on a collection MyCollection accessing its MyElement number "counter".

In Java it would be Element element = myCollection.get(counter).

In BPEL you have to toil a lot more even to do the simplest thing:

If Counter contains 4,
Temp_Variable will contain a string "/ns2:MyCollection/ns2:MyElement[4]"
which will be evaluated as a XPath by the second "copy" statement.

One can - more effectively - use dynamic indexing:

http://download.oracle.com/docs/cd/E12483_01/integrate.1013/b28981/manipdoc.htm#CIHGBJCE




Why do simple things in Java, when you can do overly complicated programming in XML?

Good introduction to BPEL

Great tutorial for absolute beginners:

http://www.radikalfx.com/bpel/language.html


Here is the BPEP XSD : http://schemas.xmlsoap.org/ws/2003/03/business-process/


The main elements of a BPEL Process definition are:

process

partnerLinks

variables

• messageType
• type
• element

sequence

• receive
• assign
• invoke (sync or async)
• reply (only for sync)

switch

while

throw

fault

empty

pick - like "receive" but lets you specify multiple triggers

flow - executes multiple activities in parallel

faultHandlers catch and catchAll




A Synchronous process will have a Receive/Reply

An Asynchronous process will have a Receive/Invoke

OSR: org.idoox.wasp.WaspInternalException: java.lang.RuntimeException: Updates to config files not supported

I get a

Error 500--Internal Server Error
org.idoox.wasp.WaspInternalException: java.lang.RuntimeException: Updates to config files not supported


after installation of OSR,
when I hit
http://localhost:7001/registry/uddi/web


After restart, I get


java.lang.LinkageError: loader constraint violation in interface itable initialization: when resolving method "com.idoox.wsdl.DefinitionImpl.getMessage(Ljavax/xml/namespace/QName;)Ljavax/wsdl/Message;" the class loader (instance of weblogic/utils/classloaders/ChangeAwareClassLoader) of the current class, com/idoox/wsdl/DefinitionImpl, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for interface javax/wsdl/Definition have different Class objects for the type javax/xml/namespace/QName used in the signature
at com.idoox.wsdl.factory.WSDLFactoryImpl.newDefinition(WSDLFactoryImpl.java:60)



It turns out that
http://forums.oracle.com/forums/thread.jspa?threadID=952237

you should not install OSR in a WLS instance that has SOA Suite stuff deployed to it.... choose an EMPTY WLS server....I will do that as soon as I find time...

Tab Kit for Firefox

Struggling with TABs in Firefox can be frustrating....
I keep closing tabs because I dislike having them disappearing at the right end side.

I have installed https://addons.mozilla.org/en-US/firefox/addon/5447 Tab Kit,
it's really nice, and it supports multiple layouts !

WSDL import vs XSD import

Ever found yourself struggling because your WSDL doesn't validate because some types - defined in an imported XSD - seem missing?

Most likely you should read this article:
http://www.ibm.com/developerworks/xml/library/ws-tip-imports.html

You better use this clause:

The clause wsdl:import should be reserved for WSDL only (although it DOES work also with XSDs), otherwise you will get the infamous:

Import of file:BLA.xsd is violation of BP 1.1 R2001.
Proceeding with a warning.
R2001 A DESCRIPTION MUST only use the WSDL "import" statement to import another WSDL description.


If you find all this confusing, so do I. At least they should give a warning at design time. Probably Java IDEs have spoiled me, I am expecting too much :o)

JDeveloper : how to clean up the list of recent Applications

Jdeveloper, how to clear application data.

Jdeveloper keeps some IDE-related data here:

C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.ide\preferences.xml
C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.ide\
windowinglayout.xml


C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.ide\projects\index2.xml
and other files in this directory



C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.jdeveloper\applications.xml

C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.jdeveloper\ide.properties



I have experimentally found that cleaning all the "Item" elements in C:\Users\pierre\AppData\Roaming\JDeveloper\system11.1.1.3.37.56.60\o.jdeveloper\applications.xml does the trick of cleaning your environment from old Applications.

Windows could not start the OracleXETNSListener service

I had the smart idea of declaring ORACLE_HOME in the System Variables, as c:\Oracle1\Middleware . Apparently this interferes with the Oracle TNS Listener installed as a System Service.
Removing the variable ORACLE_HOME from the System area fixes the problem.... funny!

Instructions to install, configure and run XMing on putty

first of all, make sure there connectivity from your server to your laptop:
log into your server and ping your laptop!


download:
http://sourceforge.net/projects/xming/

execute Xming-6-9-0-31-setup.exe

full installation (choose all defaults)

launch XMing

allow access to all networks

you should see an icon of the XMing XServer 3.0 service in the bottom right


open file X0.hosts in the XMing installation folder and add the IP of your target server (example 11.36.32.93)

restart XMing

create a putty connection and tick "Connection/SSH/X11/Enable X11 Forwarding"
login to your server (example 11.36.32.93)

export DISPLAY=11.192.1.41:0
(10.192.1.41 is your IP, you can find it with ipconfig and look at LAN IP)

to test, run "xclock" in putty

in case of problems, right-click on the XMing icon and "view log"


This error:

Xlib: connection to "11.192.1.41:0.0" refused by server
Xlib: No protocol specified

means that you haven't added the server's IP in X0.hosts

SOA suite, BPEL variables: message type, simple type, element

In a BPEL file you have a variables section, and each entry can be declared as messageType, type or element.


A variable of type "messageType" has its type declared in the "message" clause of a WSDL.

A variable of type "type" is a simple xsd type:

variable name="Counter" type="xsd:integer"

A variable of type "element" is a complex xsd type defined as element:

variable name="Books" element="ns2:BookCollection"

where BookColletion is

When interacting with Partner Links, you can only use variables of type messageType; if your data is stored in a variable of type element, you should copy it using a Assign block.

Customization File for SOA Suite

When you specify an input/output directory for your File Adapter, use "Logical Name" rather than "Physical Path" (Directory Names are specified as....).
Let's assume I specify FILES_DIR_OUT.

This will create an entry in the composite.xml



and at the beginning it's an empty string.

You should specify a token (like ${FILES_DIR_OUT}) and during your deployment process replace that token with the actual value - use any filtering mechanism like Maven Filter of a simple "sed" script....

I don't think there is a built-in support in the Ant build and deploy scripts to do that.


See also the File Adapter documentation:
http://download.oracle.com/docs/cd/E15523_01/integration.1111/e10231/adptr_file.htm#CIAHFFBI

OSB vs SOA Suite

guest post - thanks to Fabio.

SOA Suite and OSB are overlapping products, and also complementary.

Last year was it when starting another project, I did a POC to consider whether we should use SOA Suite or OSB. The project needed a bus service, so this was the only point analyzed.

A summary:

1) For long duration processes sure I would use BPEL, not OSB.

2) I have looked at using only the Mediator (the former Oracle ESB). However, I could not do what I wanted, such as exception handling and auditing. So I started to analyze BPEL

3) BPEL: the development environment in JDeveloper BPEL is much better than the OSB. Simple things like setting variables, and validate XPaths, which doesn't exist in OSB, JDeveloper SOA Suite has plugins that work very well.
However, you have to compare the performance of the OSB with the performance of BPEL. In my POC OSB was at least 15 times faster than BPEL. In my performance test, OSB reached 300 tps while BPEL is not going to 20tps. At the time I opened a call with Oracle, I disabled dehydration, and changed other settings to improve performance.

The consultants at Oracle told me I could not greatly improve the performance of the BPEL, because it was not designed to be a bus service. And this is why Oracle has chosen the OSB Service Bus strategy.

The runtime of the OSB is much more optimized than the BPEL runtime. The monitoring of OSB is also much more advanced, with real-time monitoring, SLA, alerts, etc..
When Oracle improve the plugin for OSB for something like BPEL (can be Eclipse or JDeveloper, no matter the tool), oh yes the product will be much better.



Here a great presentation on what you can accomplish with OSB and with SOA Suite, and overlapping functionalities. Especially worthy is the slide at page 13.

Of the same author, I would also recommend this presentation on testing SOA Suite environments.

SOA Suite and OSB: deploying and testing

You should create a SAR file and deploy it.

SOA SUITE:

with WLST:

some interesting WLST commands here

Here C:\Oracle1\Middleware\Oracle_SOA1\bin\ant-t2p-worklist.xml the MDS migration tool.

with ANT:
C:\Oracle1\Middleware\jdeveloper\bin\ant-sca-compile.xml

here the entire documentation on how to build and deploy using the Ant files.

OSB:

with WLST:
http://www.oracle.com/technology/sample_code/products/osb/index.html

also here another version of same script
http://www.insemble.com/oracleservicebus-deployment.html

before you run it:

c:\Oracle1\Middleware\wlserver_10.3\server\bin\setWLSEnv.cmd

set CLASSPATH=%CLASSPATH%;C:\Oracle1\Middleware\Oracle_OSB1\lib\sb-kernel-api.jar;C:\Oracle1\Middleware\Oracle_OSB1\modules\com.bea.common.configfwk_1.3.0.0.jar;C:\Oracle1\Middleware\Oracle_OSB1\lib\sb-kernel-impl.jar

java weblogic.WLST import.py import.properties

Examples:

how to compile your SCA with Ant:

cd C:\JDeveloper\mywork\myproject
ant -f C:\Oracle1\Middleware\jdeveloper\bin\ant-sca-compile.xml -Dscac.input=C:\JDeveloper\mywork\myproject\composite.xml

this will produce out.err and out.xml in your default TEMP directory
(eg C:\Users\pierre\AppData\Local\Temp)

then package into a jar file:
ant -f C:\Oracle1\Middleware\jdeveloper\bin\ant-sca-package.xml -DcompositeDir=C:\JDeveloper\mywork\myproject -DcompositeName=myproject -Drevision
=1.0

then deploy:

ant -f C:\Oracle1\Middleware\jdeveloper\bin\ant-sca-deploy.xml -DserverURL=http://localhost:7001 -DsarLocation=C:\JDeveloper\mywork\myproject\deploy\sca_myproject_rev1.0.jar -Duser=weblogic

Getting started with SVN on Windows 7

the quick an dirty way.

Download SVN from here

Install it to C:\csvn

open a CMD window
cd C:\csvn\bin
svnserve.exe -d


To IMPORT code INTO SVN:
open a CMD above the source folder you want to import
cd C:\acme\svnsource

c:\csvn\bin\svn import osb file:///C:/acme/svnrepo -m "Creation of my first project"

To CHECKOUT the code FROM SVN:

c:\csvn\bin\svn checkout osb file:///C:/acme/svnrepo osb

To provide username and password:
svn --username ${svnusername} --password ${svnpassword}

To display all projects with they last check-in info (revision, username, timestamp) use:
svn -verbose list url
example:

  12991 auser1              Oct 01 17:07 Project1/
  13636 auser2              Oct 09 13:32 Project2/
  10491 auser3              Aug 05 01:00 Project3/

To create a release notes (list of all comments for checkins since last version):
svn log url -v -r 2012-09-22:14-34 --xml

this will list all entries in a date range (from date to date)
svn log -r {2012-09-22:10-12}:{2012-09-23:10-12} That's all, folks!

Installing SOA Suite on Windows 7 64bit

Make sure you use a 64bit JDK, otherwise the installation will silently fail.

Running setup.exe with the option -jreLoc=path/to/64bitJDK might help detecting the errors.

TOAD on Windows 7: ORA-06413: Connection not open.

See http://forums.oracle.com/forums/thread.jspa?messageID=1943221

Just install TOAD in c:\Programs\TOAD  instead of the default directory (avoid any parenthesis or spaces in the folder) and IT WILL WORK! Ask no questions, just work.

It takes guts to create a folder with SPACES in its name, but it takes a real HERO to add PARENTHESIS to SPACES. Well done, Microsoft!

Mocking Web Services

SOAP UI contains a mock service generation utility.

Is it worth using it?


Luciano says:


I have used them, and for me they are complicated to maintain (we had some 20 SoapUI Mocks and it was a pain in the neck to update and deploy them to Win and UNIX).

Another problem with the mock SoapUI is linked to the fact that if you change the WSDL you realize the problem only at runtime. For us it was a problem because the WSDL would change weekly, maybe not your case.

Finally, on my advice, we've moved to "normal" Web Services deployed in a dedicated EAR. In doing so we obtained the following advantages:

- Easy to deploy (the deploy was carried out directly by Hudson, during the normal deploy the application)

- The business logic of the mock can be more sophisticated, since it's all in Java

- The mock were compiled during the normal build procedure of the project, so we could detectd problems in the WSDL at build time - rather than runtime.


The infrastructure that I had set up allowed to create a web service from a WSDL with a simple command Gradl, so the migration to "mock Java" was a matter of 3 hours work.

If you have a mock or two, with a WSDL that never changes and one environment and deploy a simple business logic, use SoapUI. Otherwise consider creating a dedicated web service.

XML Validation

When you define your Data Model, you better enforce as many restrictions as you can on your data.... too many times a request fails mysteriously because the input data fails to comply to business specifications, and the earlier you catch this - the better.

Here are the validations available in the XSD
http://www.w3schools.com/schema/schema_facets.asp


For more complex validation (like: is element E1 value is A, then element E2 can only be X or Z), you should use Schematron:

Here some links to Schematron

http://www.dpawson.co.uk/schematron/index.html
http://www.xfront.com/schematron/index.html
http://www.codeproject.com/KB/XML/schematron_started.aspx
http://www.docstoc.com/docs/2134799/Schematron-Editor-Tutorial

I am not aware of how much Schematron is supported in a Oracle "SOA Suite" project.... I shall investigate...

Getting Started with Oracle BPM Suite 11gR1 – A Hands-On Tutorial

https://www.packtpub.com/getting-started-with-oracle-bpm-suite-11gr1-hands-on-tutorial/book


The book starts with an historical overview of the BPMN and BPEL standards.
It dedicates a full chapter to installation - which is particularly nasty for this product.
It covers DBAdapter, Mediator, BPEL Process Editor, Human Workflow, Business (Jess) Rules , BAM, OWSM, Fault Management Policy

to be continued.....

Querying logs

Manually grepping logs is the single most wasteful and boring activity.

I had a look at http://www.logmx.com/ (nice videos showing main features).
Nice tool.

Yet I feel that having a Log File Query Language (LFQL? Sounds cacophonous..) similar to SQL would be much better.

I am sure I have seen something like that, if only I could remember where...

Simplify Complexity

a very interesting presentation on why we end up with overly complex architectures:

http://www.infoq.com/presentations/Simplicity-Architect

I don't agree with many technical statements (such as "HTTP is good! Transfer Objects are bad!") but the overall point is nicely presented. Plenty of interesting IT aphorisms and quotations.

VMware Fedora image doesn't connect to USB controller

VMware USB Arbitration Service service fails to start on Windows 7 with an error:

Detected unrecognized USB driver (\Driver\usbfilter).


The solution is here:

http://communities.vmware.com/thread/241662

"Try right clicking on one of the USB hubs and looking at the driver details. If usbfilter.sys is listed then open HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class{36FC9E60-C465-11CF-8056-444553540000} and delete the UpperFilter, usbfilter pair.
I rebooted and it works great now. "


I have wasted a couple of hours for this "trick".... it's working now....

still, I am downloading a ready-made fedora 14 vmware image from linuxtracker...

or rather, the version 13 from here http://www.thoughtpolice.co.uk/vmware/ since the 14 apparently doesn't support VMware Tools (?).

SNMP consoles

Fabio suggests me:

http://www.manageengine.com/products/applications_manager/application-performance-management.html - It has a web interface. It can connect to weblogic using JMX and act as a SNMP Console as well to receive SNMP Traps.

http://www.castlerock.com/ (Pure SNMP. I would not use this in productions, but it's quite handy to test receiving the traps. You can set it up in 10 minutes)

Psychodrama

Often IT floors are chronically afflicted with psychological communication barriers;
e.g. a developer doesn't talk to a manager because he is afraid of him, or 2 developers don't talk to each other because they don't feel esteem for each other.

These communication broken paths are like ischemic scars in the neuronal network of your project. They lead to impaired functionality, slower response etc.

I wonder if anybody tried reenacting those small dramas and conflicts on a stage, to laugh them off and break barriers, using techniques like http://en.wikipedia.org/wiki/Psychodrama

I am not sure what this picture has to do, but I love it:

The Hourglass Syndrome

New project, and, needles to say, new under-equipped machine.

When you run WebLogic server and Oracle Workshop (forget about Oracle DB) on a single desktop, you need definitely MORE than 2 GB.

It seems to be a STANDARD for humongous, cash-rich corporation to equip their workers with machines as HALF as powerful what a private would get from a supermarket spending 500 euros. Nobody today would buy a desktop with less than 4 GB.


The result is this: you put to sleep the creative thinking.
When you are presented constantly with an hourglass whenever you click something, it really HURTS you. It's a stress syndrome. It's a frustration syndrome (someone calls it "post traumatic wait syndrome").

You stop generating new ideas because you know that putting them in practice would put you through a lot more pain that the idea itself would bring you. If you punish with an hourglass (or an electric shock) every innovative idea, you kill innovation.

This is what corporations achieve by saving 40 Euros on a 2 GB chip of RAM.

My advice: don't expect corporations to care. Just buy a 2 GB RAM, install it and file an expenses claim.

The devastating power of rumours

In a previous life, during the performance tests of an environment, we would inject load with LoadRunner, targeting a WebLogic cluster through an Apache with WebLogic Proxy Plugin as Load Balancer.
The load generation scripts had been prepared by a LoadRunner consultant (very messy guy) who had just left the company. Funnily, this guy enjoyed good reputation, because he would prepare very good looking documents, but he was a really messy coder, copying and pasting like hell.

We would constantly measure more load on one WL instance than the other.
I asked "are we sure we target only the Apache and not the managed servers?" and they answered "absolutely".
Some guys in the corridor started spreading the rumor that WebLogic Proxy Plug-in is a poorly tested product, that it often does a poor job at balancing properly the load, and that nobody really uses it in production.
These rumors much later proved to be ABSOLUTELY FALSE, WebLogic Proxy Plugin is a ROCK SOLID product, but at the time they were enough to make everybody point the fingers against the Proxy Plugin. We spent one week with Oracle support (very patient people, I love them) trying to troubleshoot the problem, in vain.

Finally, when all hopes were lost, one chap had the brilliant idea to carefully verify the LoadRunner scripts...only to discover that they were actually often targeting one of the Managed Servers! We felt like IDIOTS in front of the Oracle Support people and our managers.

Everyone can draw his conclusion. Mine is that now I am very tough with people who spread rumors without bringing evidence. If you are not 100% sure of what you are saying, please, shut your mouth, because the consequences of your comments can be really devastating and bring about pain and losses to the company and the people around you.

Poll Result on Global Warming and Oil Spill

Time to close the poll... thanks to all of you who have voted, you are great folks.

I am quite skeptical about the result, once upon a time very few people had voted "global warming in a myth", it has made a remarkable comeback.... mmmmm.... anyway I am in London on the 4th of November and we have 13-16 degrees, as it was in mid-September....

Agile is as old as the mountains

I have read with pleasure this article on the lives of illustrious programmers.

Also available here

I found this sentence particularly intriguing:

Most of these programmers had (and have) a programming methodology that today would be called Agile. They mostly created a prototype that worked, and kept adding functionality until it was ready to ship. They worked iteratively in small teams. And, as Bricklin's current thoughts indicate, these developers were always cognizant that at some point you have to quit adding to the software and send it out the door. I found myself wondering how many readers imagine that "Agile" is something new.

Agile Methodologists try to teach us what - when left alone to self-organize our work - we have been doing for ages, out of "professional instinct" - that is the shortest path to success.

Agile, old as the mountains.

 I have listened also this presentation, very critical and well formulated. Still, I keep thinking that all these Agile practitioners are extremely verbose and religious.

Jean Tabaka and "I don't like Mondays". It was quoted by this article on Neuroscience.

This is one of the books recommended.
And this.

A Pattern Language

Amazon link

I highly recommend this book, full of original insights on Urban Planning, Anthropology and Mass Psychology.

On a general level, this article is a general presentation on the "pattern language" topic.

Java Concurrency in Practice, another book for the winter

http://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601/ref=sr_1_1?s=books&ie=UTF8&qid=1287837814&sr=1-1

I have always dealt with concurrency having a sense of uncertainty and improvisation - "just make it work with some hack" kind of thing. I really don't like this feeling. I prefer to have some good overall understanding of the most common situations one has to face, and common practice solutions.

Coders at work

I am reading this book http://www.codersatwork.com/ , it's really a great book, one of the best IT books I have ever read... probably because it's not so much about Computers, but about Computer People...

It shows how all these great coders, excellent at what they have done, have really very different personalities, educational backgrounds and lifestyles.

So far a common denominator of all these great minds is:
- learn to write clean, readable, well documented code
- learn to read other people code


Coding seems to be more a matter of literacy and communication than a matter of purely technical skills.

Here a great video on the Netscape developers, including Jamie Zawinsky

Python!

As a Python IDE I am using Oepe Eclipse 3.6

An excellent tutorial

The entire Python Standard Library

and the language specs

It looks like I will do a lot of WLST/Python... I used to do all my WebLogic monitoring using Java/JMX but it's way too heavy. I love Java but you must be pragmatic.
Python “cheat sheet” - https://www.pcwdld.com/python-cheat-sheet - it’s a quick reference to Python that I think could be useful.
It’s not a cheat sheet in the traditional sense as it’s more about helping someone to get started with Python and then providing some shortcuts to make life easier.

Play, Scalatra and their friends

I have asked Luciano:
"if you had to start a Web project from scratch, what would you use? Appfuse?"

Answer:
" Play (a Java-Groovy based Web framework)
or Lift (scala) o Scalatra (scala) "

Scala should be considered a valid alternative to Java, it's a lot less verbose...

Google Apps redirect

From today, javamonamour.org can be found also with javamylove.org

I have bought the domain from google apps, logged in the control panel

https://www.google.com/a/javamylove.org

go to Domain Settings /Domain Names / Advanced DNS settings

note your password

click on "sign in to DNS console" which brings you to

https://access.enom.com/?domain=javamylove.org&user=

on "Host records" the first entry marked @
enter the domain name you want to redirect to (javamonamour.org) and in "record type" choose "URL redirect".

Do the same on the www line, otherwise the system will only redirect when you type www.javamylove.org but not when you type javamylove.org


Easy ! :o)

IIS, introductory tour

An excellent tutorial on Installation : http://www.youtube.com/watch?v=vBNoTK31zPo

An article on the Wikipedia


Administrating IIS with scripts


The "official" IIS site: http://www.iis.net/

And the Microsoft Documentation

Some useful terminology:

Active Directory Service Interfaces (ADSI)

Windows Management Instrumentation (WMI)

Admin Base Object (ABO) interfaces



Good to know:

Application Pools


IIS Plug-In for WebLogic

NodeManager: domain salt file not found

See here the full story.

startNodeManager.cmd is in C:\bea11\wlserver_10.3\server\bin
in the same dir is also nodemanager.properties WHICH IS NOT BEING USED!

This file is configured to have:
DomainsFile=C\:\\bea11\\wlserver_10.3\\server\\bin\\nodemanager.domains


for some reason the nodemanager.domains is missing and this other is loaded:
C:\bea11\wlserver_10.3\common\nodemanager\nodemanager.domains


because NODEMGR_HOME=%WL_HOME%\common\nodemanager
where WL_HOME=C:\bea11\wlserver_10.3\

Verify the Domain name mappings:

if you are pointing to the wrong directory, where no valid domain exists, you will get this error emssage:

java.io.FileNotFoundException: Domain directory 'C:\bea11\wlserver_10.3\common\nodemanager' invalid (domain salt file not found)

Maven 3.0, welcome polyglot Maven!

at last support for Groovy POMs!

I can't believe my eyes: at last a Maven POM which is not the usual unreadable pile of XML garbage:

http://polyglot.sonatype.org/groovy.html

Here is the whole excellent InfoQ article
http://www.infoq.com/news/2010/10/maven-30-released

I am looking forward to some book on the topic...
I don't want to see a single pom.xml for the rest of my life.

The Gotha of Computing

For a history of the term "Gotha":
http://en.wikipedia.org/wiki/Almanach_de_Gotha

"The Almanach de Gotha was a respected directory of Europe's highest nobility and royalty"

James Gosling "Inventor" of Java

Grady Booch "Inventor" of UML

Edsger Dijkstra "inventor" of the Operating Systems theory

Donald Knuth researcher and author of The Art of Computer Programming

Christopher Alexander building architect often quoted for his seminal A Pattern Language

Martin Fowler writer and influencial thinker

Doug Lea father of Concurrent Java

Linus Torvalds initiator of the Linux operating system

Erich Gamma co-author of the seminal Design Patterns book

Kent Beck father of Extreme Programming and JUnit


Alan Turing, pioneer in many, many fields, not only computers - a modern times Leonardo


Tanenbaum inventor of Minix and author of a great book on Operating Systems.

Von Neumann major mathematical brain

Edgar F. Codd inventor of SQL

Dennis Ritchie, inventor of C and Unix, and his colleague Kernighan
https://en.wikipedia.org/wiki/Ada_Lovelace great mathematician who worked with https://en.wikipedia.org/wiki/Charles_Babbage on the Analytical Engine.
.... more to come ....

 (courtesy of the Dutch National Library)

Data Serialization standards.... beyond XML

I am reading a series of EXCELLENT pages on the Wikipedia:

XML

JSON

YAML


I am not saying one is better than the other. Each has pros and cons.

XML disturbs me for his prolixity, the entities like < and > and &, and the close tags . I like it because it's fairly robust and unambiguous - barred Namespaces.

JSON disturbs me for his quotes. I like its simplicity.

YAML disturbs for the reliance on indentation and special notations. I like the rigorous representation of complex data structures.


But the huge question mark raising in my head is: why on Earth do we need a format that is directly readable/editable by humans?
So that one can use Notepad or vi to edit them?

Ok, then I will give you a YAMLNotepad and a YAMLvi and we move away from a (necessarily inefficient) human readable format and use a more rigorous binary representation, mutuated for instance from Java .ser format or Coherence POF format or TIBCO or whatever other high-speed serialization standard.
We will gain in performance, validation, readibility (you can customize the program to display in whatever format you want, attaching the stylesheets of your choice)...

Why on Earth the actual data on the wire must be in the same format that we display to the user? To me it's only because it's difficult to have a standard be supported across multiple products / platform, and an ASCII editor is the most universal too available. Too bad though.... so much is lost.

Great sources of Java code

these are endless supplies of working examples:

http://www.java2s.com

http://www.koders.com/

.... more to come ....

Clean Code - a handbook of agile software craftsmanship

http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882

The PDF is also available here for free:

http://knowfree.net/2010/10/clean-code-a-handbook-of-agile-software-craftsmanship

I have started reading it and it's really enjoyable and witty - which doesn't happen very often in IT books :o(

I am reproducing on of its images, hoping  that OSNews http://www.osnews.com/ will not mind...

For a definition of Craftmanship : "emphasizes the coding skills of the software developers themselves"

Here is their manifesto, I like it http://manifesto.softwarecraftsmanship.org/

I myself believe that the main criterias to evaluate code are:
- is it readable and easily modifiable?
- are build, test and deployment fully automated with a CI tool?
- is it fully covered by multithreaded automated tests?

One more thing I have to say: I am appalled that 90% of the times automated tests are executed with a single client thread... I have seen too many times multithreading issues surface during the first phases of a live application.

Java Collections, some important facts

Here the main entry point, with an excellent tutorial by Joshua Bloch.

Collections can be: Set (TreeSet, HashSet), List (Vector, LinkedList), Queue (LinkedList), Map (HashMap, TreeMap, LinkedHashMap)

Some implementations are Synchronized (Vector), other no (ArrayList)

Collections provide Enumeration or Iterator to scan them.
Use Iterator when you might want to make changes to the collection, use Enumeration when R/O. You have also ListIterator

java.util.concurrent package provide thread-safe optimized implementation of most collections.


ArrayList and Vector use a dynamic array as an implementation,
LinkedList uses a doubly linked list, and it's faster for insert/delete

Queue: FIFO
Stack: LIFO



Interview questions here:
http://www.interview-questions-java.com/java-collections.htm

Europass CV, how to write your CV in a standardized way

I have been thinking of making a "live, searchable CV" using some a data representation format (NOT XML that I hate) and some Groovy report thingie...

I am just tired of having to answer all those questions "how may years of WebLogic or EJB do you have?" ... just annotate each project with the technologies used an run a report like:

select sum(p.duration), p.technology_used from myprojects p group by technology_used;

My sister told me that in EU this is the Europass CV standard (funny that nobody seems to enforce it in the IT recruiting world, and each agency comes up with its own model, so that they are completely non-interchangeable and you have to keep duplicating your effort... thanks god we are the IT specialists! )

https://europass.cedefop.europa.eu/europass/home/hornav/Downloads/TechnicalResources/XML.csp

I am planning to investigate it and eventually extend it for my purposes.

Also, to produce a standardized report of you languages (I mean English, French...) knowledge, use this standard:

http://europass.cedefop.europa.eu/europass/home/vernav/Europass+Documents/Europass+Language+Passport.csp

How to produce your own video with CC (closed captioning)

Just investigating the options...

A free Screen Capture tool is http://camstudio.org/ , very nice.
He sells this http://www.podcastautocue.com/ tool to annotate your videos...

This are instructions on how to closed captioning (CC) a video on youtube:
http://www.youtube.com/watch?v=kfPW9uXPQPA&feature=related

the online tool looks a bit cumbersome to me, too many manual hacks.

This solution http://captiontube.appspot.com is also available, and seems to me simple and cool.

A paid solution is Sony Vegas Movie Studio, it's only a few bucks:

http://www.sonycreativesoftware.com/moviestudiohd


There are 2 formats: SubRip and SubViewer... SubViewer 2.0 seems better because it supports line breaks.

I have seen people using "BB flashback", they seem happy. It's a paying product though.

Video killed the Email star

Yesterday I have seen a demo of http://screenr.com/ , you create a video and email or tweet it, the video stays archived.

The person who has shown this to me is a Pakistani IT entrepreneur, http://www.rehan.com/ , he says he does all his communication on videos, instructions to his collaborators and especially training.

His employees are asked to use videos for everything, so when one leaves another can watch the videos he left behind and catch up.

No communication is private, everything is public and archived.

Not everybody like this, in fact this diminishes the power of the individual and make people more replaceable - which is sad from the individual perspective but good for the company.

In fact, I believe that at least 30% of IT projects problems stem from Information Retention from people who keep things as less documented as possible to increase their personal influence. But if you say so out loud you will not be very popular.

He also showed me Khan Academy, these people are great, BTW they need help so if you have spare time you can assist them:



I love the idea of creating YouTube tutorials for Java, WebLogic and WebServices.
I see that even Oracle has started putting videos on YouTube.

And now, enjoy this wonderful song:



we can't rewind we've gone too far!

The Beauty of JOIN(T)S

I don't think that the majority of Java Developers know that
( see http://en.wikipedia.org/wiki/Join_%28SQL%29 )
in nature there is an incredible number of JOINs :

INNER JOIN
IMPLICIT JOIN
EQUI JOIN
NATURAL JOIN
CROSS JOIN
LEFT OUTER JOIN
RIGHT OUTER JOIN
FULL OUTER JOIN
SELF JOIN

Of course if you think of a JOIN in terms of SEX, there are a lot of Kamasutra positions here to try (I am curious about the EQUI: does it refer to a horse? Jeez! Anyway I am an expert in SELF JOIN).


Most likely you will never use ALL of them, but when you struggle with a wrong result set it's good to know that you have many options to try - and then give up and start using a ORM tool.

And here is a NORMAL JOIN

and instructions on how to use it: http://www.seedsfromamsterdam.com/joint.html
(long live Amsterdam, one of the most beautiful cities on Mother Earth)

SOA stuff to keep you warm in winter

especially if you shove them in the stove :o)

A general overview here:
http://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29


Plenty of patterns available here:
http://www.soapatterns.org/


Plenty of podcasts here:
http://itunes.apple.com/podcast/onsoa/id278745478

And the usual book to read before you go to bed:

http://www.amazon.com/Design-Patterns-Prentice-Service-Oriented-Computing/dp/0136135161

I am reading the Thomas Erl book, I am not overly impressed, I think the same things could be said in much less and simpler words.

Buy that hardware

As a consultant, I am often faced with starting a new project and being assigned insufficient hardware.

And spending anything between 10 and 30% of your time watching an hourglass.

And having painful context switches when trying to do something else while waiting.

And accumulating neurosis.

My experience is that, most of the time, your complaints will not be heard. Or they will be taken care of 2 month later, when the damage will already be humongous.

I never understood why managers don't realize how vital it is that developers have top notch equipment. I assume that this is because all they do is IE, Word and Excel and they are never faced with running Eclipse, XMLSpy, Word, Firefox, OracleXE and a WebLogic Cluster on the same desktop.

Some of them come up with funny recommendations like "why do you code in Eclipse, you can use Notepad instead". Or "why can't you deploy your code to the integration box". Getting mad will lead you nowhere. Stay cool.
This is the way things work in IT. Nobody would ever recommend a surgeon to use a kitchen knife instead of a scalpel, but in IT everything can happen.

So, my only recommendation is: buy your own hardware and develop on it. You can buy a top box with the money you make in 2 days. Or your extra 4 GB RAM with half a day salary. You can still sell it at the end of the project.

No point in fighting windmills. Find your way around.

The Developer turns into an Historian

So you are a developer and you have just received an assignment.

If you work in a normal IT project, this means that someone sent you an email saying "implement me a system which processes credit card transaction", with a couple of documents in attachment explaining the structure of the records.

If you are lucky, you might get a link to some SVN repository where some developer who has already left the company wrote some undocumented code trying to do the same; they will tell you "yes but this code was not good. Now you are an Agile developer and you will figure out what to do".

If you are a normal human being, you will now have a sense of inadequacy and immense responsibility weighing on your shoulders. And a sense of panic, because the work of 4 people (the business analyst, the architect, the developer and the tester) has been entirely pushed to you, without you having the status/authority in the company to go around and demand that some analysis and design is done.

As a developer, you are at the bottom of the company ladder and your only option is to say "yes" (do I sound "victimistic" (=with a persecution complex)?) .Well, no, you have an option to say "no" and you should exercise it, for the good of the project.

Being poorly specified, the problem will be necessarily poorly implemented, this will imply rejection at delivery, and successive expensive reworks to get it done right. And everything will be the developer's fault.


My point is: as an experienced developer, your duty is to stand up and say "these requirements are insufficient, we cannot proceed to implementation unless we go through a phase of debate, analysis and design, and we reconstruct the previous attempts that were done on this problem, to learn from the past and gather all the useful information from the people who were involved". It is your duty to say "let me talk to some business analyst, to get the entire picture and document it".

All issues have a history behind. It is important to track this history into a central place. Letting things to stay dispersed is a door open for chaos. They only way to successfully complete your assignment is through full control and tracking. Keeping all the info in a Wiki is indispensable. Better still if you can keep a (b)log of what has been done and needs to be done, with the participants clearly indicated. Keep all participants informed. Blogs and RSS feeds can be useful, although simple emails can do.


Being a good developer is not only about being techie techie... most of the time, it means having a relentless commitment to investigation, communication and documentation. The same skills it takes to be a good historian.

Once I have been asked to perform performance tests on a system. I worked one month setup the environment, to prepare all the Python scripts, measure the metrics, investigate the memory leaks.... by chance, on a Saturday, speaking informally to a colleague sitting 5 meters from me, I discovered that he had done exactly the same thing 3 months before, and already come to a lot of diagnostic. Why management didn't tell me before? Probably they will say "because you didn't ask". This is the world, you cannot change it. Getting mad will lead you nowhere. So, next time, ask "has anybody been working on this before"? There are no stupid questions, you are stupid if you don't ask.