Puppet: file recurse remote caveat

Puppet contains a beautiful "macro" file recurse remote, by which you keep a while directory (with subdirectories!) in sync with the content of a Puppet files repository. Great.

BUT. There is a BUT: if apart from maintaining the whole folder (say: /myfiles) with recurse-remote, you also maintain a file in a subfolder (say: /myfiles/myfolder/hello.txt), this breaks the whole synchronization of /myfiles/myfolder/. OTHER subfolders (say: /myfiles/myotherfolder/) will syncronize perfectly, but not /myfiles/myfolder/.

Now you know, so you can plan your workarounds to this - maybe unexpected - behavior.

Java JVM Flag PrintConcurrentLocks

When all hell break loose and you have threads hanging waiting for a lock, it's PARAMOUNT to be able to determine WHO is holding that lock.


By default, the option -XX:+PrintConcurrentLocks is not enabled. if you enable it, and you do a "kill -3 PID", you should get for each thread the list of locks being held. This option "should be" safe in PROD, despite of the Oracle warnings.

However, you can get the same info using

jstack -l PID

"Reference Handler" daemon prio=10 tid=0x000000004e8b6000 nid=0x48c8 in Object.wait() [0x000000004104b000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000782f36550> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x0000000782f36550> (a java.lang.ref.Reference$Lock)

   Locked ownable synchronizers:
        - None

The "Locked ownable synchronizers:" bit is the one you would get extra by using the PrintConcurrentLocks flag.

Anyway this is only a superficial analysis... luckily working with OSB I never had to deal with locking issues... so I cannot claim to be an expert.

Installing a NFS server, and creating a mountpoint, in 30 seconds

This on RHEL/CentOS. For lazyness simplicity I install client and server on the same box :o) whose IP is 10.0.2.15

Installing a NFS server:
http://www.howtoforge.com/setting-up-an-nfs-server-and-client-on-centos-5.5

yum install nfs-utils nfs-utils-lib
chkconfig --levels 235 nfs on
/etc/init.d/nfs start

mkdir /var/nfs
chmod 777 /var/nfs
vi /etc/exports
/var/nfs        10.0.2.15(rw,sync,no_subtree_check)

exportfs -a

Creating a mountpoint
mkdir /home/users
mount -t nfs 10.0.2.15:/var/nfs /home/users
echo ciao > /home/users/ciao.txt
cat /var/nfs/ciao.txt
ciao

done :o)

Useful links:
https://www.suse.com/communities/conversations/configuring-nfsv4-server-and-client-suse-linux-enterprise-server-10/
http://www.javamonamour.org/2014/10/mount-nfs-and-rpcbind.html

Book: Spinoza, a life

Brilliant biographical and historical book, splendidly depicting the Netherlands and the Jewish community in Holland in the 1600-1700 period.

Steven Nadler is one of the top world experts in Spinoza, and he gives an image of this great intellectual which is much more multifaceted and dynamic than the traditional image - tending to portray Spinoza as a very secluded and shy man. On the contrary, he had a very active social life and he was a great innovator in a lot of realms, mainly of Optics, Astronomy and Biology.

Book: Unveiling India

This book tell stories of "ordinary" (actually, extraordinary) Indian women, in their daily struggle in a very sexist country.

The book was written in 1987 and things have changed a lot, however it's a wonderful testimony or real life, interviewing hundreds of women - mostly from low-income class - across the whole India.

"In a dust-filled yard I meet an old woman bent over a pile of dry palm leaves. 'Child, don't sit on the ground. Let me spread a mat for you,' she says without looking up, pulling from under her a tattered mat which I realize is the only one she owns. She is eighty years old and lives alone. Her three sons are married and have gone away to bigger villages. Who looks after her? She points to the pile of palm leaves and goes back to cleaning them."

Puppet: mount a SAMBA share at boot

First, create a credentials file in your acme module:
acme\files\samba\myshare.smb.credentials
and inside put:

username=myuser
password=mypassword

I was unable to make it mount an unprotected share..... so just have it protected by username/password.

Then in your acme init.pp read a boolean

$acme_install_sambamyshare = any2bool(hiera('acme_install_sambamyshare', false)),

Create a samba.pp class (file):

# == Class: acme::samba
#
# Manages samba mounts
#
class acme::samba {
  if $acme::acme_install_sambamyshare {
    file { '/data/myshare/':
      ensure => directory,
      owner  => "soa",
      group  => 'soa',
      mode   => '0775',
    }

    file { '/etc/myshare.smb.credentials': source => 'puppet:///modules/acme/samba/myshare.smb.credentials', 
  owner => root,
      mode => '0700', } ->
    mount { 'myshare':
      name    => '/data/myshare/',
      atboot  => 'true',
      device  => '//windowshost/sharedfolder',
      ensure  => 'mounted',
      fstype  => 'cifs',
      options => "credentials=/etc/myshare.smb.credentials,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777",
      require => [Package['samba-client'], Package['cifs-utils'], File['/data/myshare/'], File['/etc/myshare.smb.credentials']],
    }
  }

}

Don't forget to declare the samba class in your init.pp!

If you do cat /etc/fstab you should see something like this:

//windowshost/sharedfolder /data/myshare/ cifs credentials=/etc/myshare.smb.credentials,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0

If you want to do the same thing manually, this is the recipe:

yum install smbclient
//yum install cifs-utils for Ubuntu
/bin/mount -t cifs -o username=someuser,password=somepassword,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 //windowshost/sharedfolder /data/myshare/

To remove a mountpoint, use umount:

umount /data/myshare/

it supports also a -f (force) option:

umount -f /data/myshare/

See also here

OSB: extending a domain by adding extra Managed Servers

See Oracle document "WLST Script to Add a Managed Server to an Existing OSB Cluster (Doc ID 1450865.1)", the provide you a WLST script extendOSBDomain.py to do the job.

The script must:

• Create the new MS and set its listen-address and port etc etc
• Alter the cluster to add the new MS to the cluster address
• Create the JMS resources ["FileStore", "WseeFileStore"] (filestore, Jmsserver), creating also the $DOMAIN_HOME/WseeFileStore_auto_* etc folders, the JMS module and queues ["wli.reporting.jmsprovider.queue", "wli.reporting.jmsprovider_error.queue", "wlsb.internal.transport.task.queue.email", "wlsb.internal.transport.task.queue.file", "wlsb.internal.transport.task.queue.ftp", "wlsb.internal.transport.task.queue.sftp","QueueIn"]
• create a SAF agent

So, on the whole, the job is not as simple as "clone an existing MS". incidentally, cloning doesn't reproduce exactly all settings of the original MS, for instance, log properties like "RotateLogOnStartup" and "limit number and size" are NOT copied, nor are logfilters (and forget about filestores, JMS resources etc).

See also this doc about scaling up.

Puppet: quick hiera setup

cat /home/soa/.puppet/hiera.yaml

:hierarchy:
  - common

:backends:
    - yaml

:yaml:
    :datadir: '/home/soa'

cat /home/soa/common.yaml

install_cleanssbatchorder : true

To test if hiera is working:

puppet apply -e "if hiera('install_cleanssbatchorder', false) { notify { 'pippo':}} "

this should print:

notice: pippo
notice: /Stage[main]//Notify[pippo]/message: defined 'message' as 'pippo'
notice: Finished catalog run in 0.02 seconds

Puppet: minimalistic custom fact

in your module, create a lib/facter folder.
There, create a customxpathversion.rb file containing this code:

require 'puppet'

Facter.add("customxpathversion") do
  setcode 'cat /opt/oracle/scripts/config/customxpathversion.txt'
end

where the file customxpathversion.txt contains "1.10"

Now if in your puppet code you do

notify { "${::customxpathversion}" : }

you get 1.10

Of course one should handle errors etc. But it works and I don't have to bother about $LOAD_PATH. From command line "facter customxpathversion" will not work because probably it's not in the LOAD_PATH.
Here the whole documentation Update: you can use this custom fact from the facter command line if you set the FACTERLIB environment variable:

find / -name customxpathversion*

/tmp/vagrant-puppet-3/modules-0/pippo/lib/facter/customxpathversion.rb
^C
[soa@osb-vagrant ~]$ export FACTERLIB=/tmp/vagrant-puppet-3/modules-0/pippo/lib/facter/
[soa@osb-vagrant ~]$ facter customxpathversion
1.10

WebLogic: Installing WebLogic 10.3.5 binaries and creating a domain in 2 minutes

Sometimes one need to create a throwaway domain in minutes....I hate having to remember all the small details....
how to create a weblogic domain
login to myhost
sudo su - soa
cd /opt/oracle/software
make sure wls1035_generic.jar is there
type "java -d64 -version"
if this gives 'java version "1.6.0_33"' -> OK


vi silent.xml

<?xml version="1.0" encoding="UTF-8"?>
   <bea-installer>
     <input-fields>
       <data-value name="BEAHOME" value="/opt/oracle/fmw11_1_1_5" />
       <data-value name="WLS_INSTALL_DIR" value="/opt/oracle/fmw11_1_1_5/wlserver_10.3" />
       <data-value name="COMPONENT_PATHS"
value="WebLogic Server/Core Application Server|WebLogic Server/Administration Console|WebLogic Server/Configuration Wizard and Upgrade Framework|WebLogic Server/Web 2.0 HTTP Pub-Sub Server|WebLogic Server/WebLogic JDBC Drivers|WebLogic Server/Third Party JDBC Drivers|WebLogic Server/WebLogic Server Clients|WebLogic Server/WebLogic Web Server Plugins|WebLogic Server/UDDI and Xquery Support|Oracle Coherence/Coherence Product Files|Oracle Coherence/Coherence Examples"/>
       <data-value name="NODEMGR_PORT" value="5556" />
       <data-value name="INSTALL_SHORTCUT_IN_ALL_USERS_FOLDER" value="no"/>
    <data-value name="LOCAL_JVMS" value="/usr/lib/jvm/java-1.6.0-sun-1.6.0.33.x86_64"/>
   </input-fields>
</bea-installer>

java -d64 -jar /opt/oracle/software/wls1035_generic.jar -mode=silent -silent_xml=silent.xml
alias wlst="/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/bin/wlst.sh"
vi createToolsDomain.py

readTemplate('/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/templates/domains/wls.jar')

cd('Servers/AdminServer')
set('ListenPort', 7101)
set('ListenAddress','myhost.acme.com')
cd('/')
cd('Security/base_domain/User/weblogic')
cmo.setPassword('bla')
setOption('OverwriteDomain', 'true')
writeDomain('/opt/oracle/domains/toolsdomain')
closeTemplate()
exit()

run this:
wlst createToolsDomain.py
cd /opt/oracle/domains/toolsdomain
vi start.sh
nohup /opt/oracle/domains/toolsdomain/startWebLogic.sh > /opt/oracle/domains/toolsdomain/adminserver.out 2>&1 &

chmod 755 start.sh
Console in http://myhost.acme.com:7101/console/

WebLogic : awesome presentation on what is new in 12.1.3

http://download.oracle.com/tutorials/ecourse/fmw/wls/wls_12.1.3_new_features/presentation.html

I VERY HIGHLY recommend to go through this 2 hours presentation on the new features of WebLogic 12.1.3.

you will learn - amongst other stuff - about:

• dynamic clusters and server templates
• automatic server migration
• REST interface for Administrative commands - instead of WLST (awesome!)
• Exalogic (replicated store, cooperative memory management, JDBC optimizations)
• Transactions
• JMS (integration with AQ JMS, migrateable servers)
• Security: separate keystore per channel, Virtual Users with X509
• Classloader Analyzer Tool (CAT)
• Managing resources with Enterprise Manager

Books: The Politics of Heroin, The History of the First World War

Just finished reading these 2 MASSIVE history books, the first tells a lot of really interesting stories on how geopolitical interests control the production and traffic of heroin in the world. Most of the book is about the Golden Triangle (Thailand, Laos, Myanmar) and it really helps to read at least the Wikipedia about these 3 countries before reading this book, because things are really intricate.

http://en.wikipedia.org/wiki/The_Politics_of_Heroin_in_Southeast_Asia

It's definitely a book you need to read twice to digest it.

This one is a very comprehensive coverage of the military social and political events before, during and after WW I. This war shaped the world and laid the foundations for WW II, so it's not really "old stuff", it's an account of how modern warfare got organized in an industrial way of killing.

The author is very well documented, and he exposes very well the subject. However I got the impression that he underestimates the role that International Finance had in preparing the war - my personal opinion is that events of that caliber are programmed several years in advance by the International Elites to reshape the world to their advantage, and to pile up massive profits in war-related business. USA intervened at the last minute, played only a relatively minor role, yet they almost unilaterally dictated the peace conditions - with great irritation of UK and France. They played basically the same trick in WW II. Hence I would really like to understand the role they played in preparing the war - even if they always pretended to be completely isolated from the Old Continent.

WebLogic: moving JMS filestores around

If a domain dies, and the filestores contain precious JMS messages still to be processed, how do you handle this?

There is a "filestore export" utility, but sadly it doesn't have an equivalent "import" functionality.

We just made a silly experiment - silly because this is a total hack and unsupported by Oracle, yet it seems to work.

Shutdown both source and destination Managed servers. Physically replace the destination .DAT filestore with the source filestore (each managed server has its own filestore....so you have to make this operation for all filestores=managed servers) . Start the destination server. By miracle the JMS messages appear on the destination server. Of course you lose all the previous content of the destination filestore...

WebLogic: WLST timeout connecting on a secure port

Connecting to a WebLogic server on a SSL port, although highly recommended by Oracle, can have a major drawback: it's damn slow to establish connection.

AFAIK there is no way to speed this up. I have tried the customary "urandom" optimization, to no avail.

So I had to increase the "SSL Login Timeout" (server/configuration/tuning).

"Specifies the number of milliseconds that WebLogic Server waits for an SSL connection before timing out. SSL connections take longer to negotiate than regular connections."

in WLST:

serverName = 'osbts1do'
cd('/Servers/' + serverName + '/SSL/' + serverName)
cmo.setLoginTimeoutMillis(30000)

Python: run multiple processes in parallel and wait for completion

This code is quite minimalistic but it works really well. It employs a very old API which is compatible with Pythong 2.1 (WLST very old version of Python....)

from sets import Set

processes = Set()

for command in commandlistlist:
    print "running", command
    processes.add(os.popen(command)) 

#wait until all processes have completed
for proc in processes:
    proc.read()

sqlplus vs sqldeveloper

When I run a DDL script in SQLDeveloper, it just runs perfect. Same script in SQL throws plenty of errors like:

ORA-00955: name is already used by an existing object

Incidentally, SQLPlus runs me completely crazy because it never tells you in a single line which TABLE is actually giving trouble, so it becomes really difficult to grep and report for errors...

Anyway, it turns out that if you run this in SQLPlus:

  CREATE TABLE "BUILD_POINTS" 
   ( "ENV" VARCHAR2(2 BYTE), 
 "SUCCESS" NUMBER, 
 "POINTS" NUMBER
   ) ;
/

it interprets the final / as a "repeat last command", which obviously fails because the table was already created.

The problem is that I generate this DDL with a SQLDeveloper export, so I have little control on how this is being generated. I cannot unconditionally remove all / , because in the "create package" statement they are absolutely necessary. I should write a parsing script to conditionally remove the / when they belong to a "create table".

Being able to edit tables in SQLDeveloper and then export them is too convenient....I don't want to resort to having to code SQL manually...

One workaround could be inserting a table comment for each table, so that the / would be inserted after the "COMMENT ON TABLE" statement and not after the "CREATE TABLE" statement. Repeating a "COMMENT ON TABLE" doesn't generate any error.

See also this and better still this

An excellent solution is to upgrade to SQLDeveloper 4, which doesn't generate the ";\" sequence.

Check also the (quoting) checkbox called "Terminator" which when selected uses semicolons to terminate each statement (unquoting). It's in Tools/Database/Utilities/Export.