Puppet and nested hashes in hiera

In Puppet, representing your data like a normal mortal would do leads you nowhere.
You must model your data in a way that Puppet can access.
This is driving me absolutely crazy, in Puppet there are only straight lines and if you want to bend, you must compose 100 straight lines at a small angle.
So, this highly normalized representation of an array of keystores is not usable by Puppet:

---

keystores:
  - keystorename : dev.jks:
    type : identity
    password : 111111
    certificates: 
      - name : pippo.cer
        alias : pippo   
      - name : pluto.cer
        alias : pluto      
       
  - keystorename : trust.jks
    type : trust
    password : 111111
    certificates: 
      - name : pippo.cer
        alias : pippo   
      - name : pluto.cer
        alias : pluto   

because Puppet cannot extract an array of "keystorename" from it. You can do:

$keystores = hiera('keystores')

#this generates a String
$keystorename = $keystores[1]['keystorename'] 

but you cannot say

#this generates an Array
$keystorename = $keystores[]['keystorename']
#FAILS!!!

Nor you can access the element of the array which has a given keystorename. I really miss here the query capability of XPath. There doesn't seem to be a XPath for YAML.
Look on Puppet hash documentation and you will find almost no help.

The workaround is to denormalize my data, using named instances and listing these names in a separate array keystorefiles:

---
keystorefiles:
  - dev.jks
  - trust.jks

keystores:
  dev.jks:
    type : identity
    password : 111111
    certificates: 
      - name : pippo.cer
        alias : pippo   
      - name : pluto.cer
        alias : pluto     
       
  trust.jks:
    type : trust
    password : 111111
    certificates: 
      - name : pippo.cer
        alias : pippo   
      - name : pluto.cer
        alias : pluto      

and then you can access the data:

$keystorefiles = hiera('keystorefiles')
create_jks_store {$keystorefiles:}

define create_jks_store {
  $keystores = hiera('keystores')
  $keystore = $keystores[$name]
  notify { $keystore['type']:}  
}

notice the trick $keystore = $keystores[$name] to retrieve the individual named instance (dev.jks or trust.jks).

It takes a lot of effort to twist your brain to think this way.

Puppet: setting up hiera configuration in Vagrant

In Puppet, hiera configuration file is expected to be in /etc/puppetlabs/puppet/hiera.yaml.
How to automate the copy of this file, so that every time you do a "vagrant up" the whole Puppet environment is ready?
This is one way: in your Vagrantfile, BEFORE the " config.vm.provision :puppet do |puppet|" clause, you add this: config.vm.provision :shell, :inline => "ln -sf /vagrant/configuration/hiera.yaml /etc/puppetlabs/puppet/hiera.yaml"
and in your Vagrant folder on your host - under the folder containing the Vagrantfile - you create a folder "configuration", containing a hiera.yaml:

---
:backends: yaml
:yaml:
  :datadir: /vagrant/configuration/hieradata
:hierarchy: 
  - osb-vagrant-local
  - osb-vagrant
  

and a subfolder configuration/hieradata containing the files osb-vagrant.yaml and osb-vagrant-local.yaml (osb-vagrant is the hostname of my box as in config.vm.host_name = "osb-vagrant.acme.com"
Vagrant will "mirror" to /vagrant on the guest whatever is in your current folder on the host. The symbolic link will do the rest.
At this point you can enter any variable in osb-vagrant.yaml, like

pippo : pluto

and if you do hiera('pippo) you should get pluto.

Puppet: create files from Array

Suppose I have such entry in a yaml file.

certificatefiles:
   - Cert1.cer
   - Cert2.cer

and I have all those *.cer files in mymodule/certs/ folder in Puppet.
I cannot do:

mycertificates = hiera('certificatefiles')

for mycertificate in $mycertificates :
    file {"/opt/oracle/certs/${mycertificate}":
       ensure => present,
       source => "puppet:///modules/mymodule/certs/${name}",
    }

because if would be too intuitive and simple, and Puppet bans whatever is fluent and simple. Iterations are not supported, because Puppet believes Iterations are a bad concept (who need iterations by the way, when you can hardcode and copy/paste to death?).

This is the VERY ELEGANT Puppet approach (very intuitive, it took me days to figure it out):

define create_certificate_file($target_folder) {
  
  file { "${name}":
    ensure  => present,
    path    => "${target_folder}/${name}",
    source  => "puppet:///modules/mymodule/certs/${name}",
  }
}

mycertificates = hiera('certificatefiles')

create_certificate_file {$mycertificates :
    target_folder => '/opt/oracle/certs/',
}

$mycertificates is an ARRAY, and Puppet has decided that whenever you invoke a define passing an array in the "name" field, it means implicitly that you want to repeat the define for each element of the array.
BE VERY CAREFUL not to write
invoke create_certificate_file {"${mycertificates}":
this would transform your array into a String "Cert1.cerCert2.cer" which defeats the purpose.

I hope this will save hours of frustration to the next poor beast falling victim to Puppet DSL.
PS incidentally, who needs Types? Life is good when you can implicitly convert an Array into a String, and waste hours troubleshooting the issue.... who need Objects and Classes? let's represent everything with a String and Hashmaps... let's be FLEXIBLE like molassa and spaghetti.

Unable to find interpreter named WebLogic 10.3.5 WLST

If you get this error message
Unable to find interpreter named WebLogic 10.3.5 WLST
in an Eclipse project containing WLST code, chances are that you forgot to configure a Runtime.
Windows/Preferences, search for "runtime", add an Oracle runtime pointing to your local installation of WebLogic, and wait for Eclipse to reconfigure WLST interpreter.

OSB strange blocked threads while hammering a server

I was pounding with requests a service with TRACING and LOGGING enabled, at debug level. This seemps to create a lot of contention. Avoid using tracing in any critical environment (prod).

"[ACTIVE] ExecuteThread: '82' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaacc3cc000 nid=0x1086 waiting for monitor entry [0x000000004b404000]
   java.lang.Thread.State: BLOCKED (on object monitor)
 at org.apache.xmlbeans.impl.store.Path.getCompiledPath(Path.java:136)
 - waiting to lock <0x0000000767e381f0> (a java.lang.Class for org.apache.xmlbeans.impl.store.Path)
 at org.apache.xmlbeans.impl.store.Path.getCompiledPath(Path.java:130)
 at org.apache.xmlbeans.impl.store.Cursor._selectPath(Cursor.java:902)
 at org.apache.xmlbeans.impl.store.Cursor.selectPath(Cursor.java:2634)
 - locked <0x00000007f8276f38> (a org.apache.xmlbeans.impl.store.Locale)



"[ACTIVE] ExecuteThread: '61' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aaaba01d800 nid=0xa59 waiting for monitor entry [0x0000000049ff0000]
   java.lang.Thread.State: BLOCKED (on object monitor)
 at com.bea.logging.RotatingFileStreamHandler.publish(RotatingFileStreamHandler.java:75)
 - waiting to lock <0x0000000797c61b08> (a weblogic.logging.FileStreamHandler)
 at java.util.logging.Logger.log(Logger.java:478)
 at com.bea.logging.BaseLogger.log(BaseLogger.java:51)
 at weblogic.logging.WLLogger.log(WLLogger.java:40)
 at com.bea.logging.LoggingService.log(LoggingService.java:250)
 at com.bea.wli.sb.messages.ServiceLogger.enterPipelinePair(ServiceLogger.java:1693)
 at com.bea.wli.sb.pipeline.LoggingTracingStep.traceEnter(LoggingTracingStep.java:211)

to be investigated.... If you find the same issue, please scream.

sudo Runas_list (sudo by example)

I have 3 users "soa", "soa2", "soa3". I want to allow a user "myuser", member of the group "support", to run ANY script in /opt/oracle/usr folder, but ONLY impersonating the soa, soa2, soa3 user - the script should never been run as myuser user.

I create a group "support" and a user myuser:

puppet apply -e "group {'support': ensure => present,}
puppet apply -e "user {'myuser': ensure => present, gid => 'support', managehome => true}"

then I edit my sudoers file:

visudo -f /etc/sudoers

and I enter

Runas_Alias SOA = soa, soa2, soa3

%support osb-vagrant= (SOA) /opt/oracle/usr/*

"%support" means "any member of the support group"
"osb-vagrant=" means "only when the operator is logged on this hostname"
"(SOA)" means "only if the operator runs the command with a sudo -u soa|soa2|soa3 command"
"/opt/oracle/usr/*" means "operator can run any script in /opt/oracle/usr/ directory"

If you want to allow the operator to run the command without having to supply his password:

%support osb-vagrant= (SOA) NOPASSWD: /opt/oracle/usr/*

now if you "sudo su - myuser" and run "sudo -u soa /opt/oracle/usr/somescript.sh", you will be able to run the script impersonating the user "soa".

sudo is incredibly flexible

sudoers file syntax (sudo by example)

Here the official (?) doc for sudoers and sudo. Particularly important in the sudoers file format. It's very important to read all the options, since it's very versatile and allows you to get well organized if you know them.

The sudoers file format is:

"user list" "host list" = "operator list" "tag list" "command list"
"tag list" is the NOPASSWD / PASSWD option, and it's optional.
where each list can be made of actual values or aliases (User_Alias Runas_Alias Host_Alias Cmnd_Alias)

A simple example here: how to give all root privileges to user vagrant:

Login as root
visudo
append this:

vagrant    ALL=(ALL)       ALL

exit visudo with :wq
sudo su - vagrant
cat /etc/sudoers

cat: /etc/sudoers: Permission denied (of course, vagrant user cannot see the /etc/sudoers file which is only read for root)

sudo cat /etc/sudoers

this one works, because it's being executed as the root user !

Hint: you can also use visudo -f /etc/sudoers.d/filename

Puppet 3.2 graph and cycles

Awesome article on this topic.

To generate a graph, simply type this:
puppet apply --modulepath=/tmp/vagrant-puppet/modules-0/ osb-vagrant.pp --noop --graph

In case (it happens often!) you have a dependency cycle ( A -> B -> A ), you will get the dreaded message:

[root@osb-vagrant manifests]# puppet apply --modulepath=/tmp/vagrant-puppet/modules-0/ osb-vagrant.pp --noop --graph
err: Could not apply complete catalog: Found 1 dependency cycle:
(File[/etc/sudoers.d/nesoav2] => Class[Nesoav2::Sudo] => File[/etc/sudoers.d/nesoav2])
Cycle graph written to /var/opt/lib/pe-puppet/state/graphs/cycles.dot.
notice: Finished catalog run in 0.32 seconds

This is new compared to previous editions. Previously the whole resource graph was generated, now only the cycle is written to the file - making investigation a lot easier.

How do I generate graphs?

 puppet --configprint confdir

/etc/puppetlabs/puppet

vi /etc/puppetlabs/puppet/puppet.conf

[main]
    vardir = /var/opt/lib/pe-puppet
    logdir = /var/log/pe-puppet
    rundir = /var/run/pe-puppet
    modulepath = /etc/puppetlabs/puppet/modules:/opt/puppet/share/puppet/modules
    user = pe-puppet
    group = pe-puppet
    archive_files = true
    archive_file_server = puppet.acme.com

[agent]
    certname = localhost.localdomain
    server = puppet.acme.com
    report = true
    classfile = $vardir/classes.txt
    localconfig = $vardir/localconfig
    graph = true
    pluginsync = true

note the "graph = true " in the agent section of puppet.conf file.

To check the directory where the graph files will be written:

puppet --configprint graphdir
/var/opt/lib/pe-puppet/state/graphs


 ls -ltr /var/opt/lib/pe-puppet/state/graphs
total 120
-rw-r--r-- 1 root root 28078 Sep 21 22:20 resources.dot
-rw-r--r-- 1 root root 26513 Sep 21 22:20 relationships.dot
-rw-r--r-- 1 root root 54997 Sep 21 22:20 expanded_relationships.dot
-rw-r--r-- 1 root root   147 Sep 21 22:20 cycles.dot

so all these files will be generated every time I run "puppet apply"

just run "dot -Tpng resources.dot -o resources.png" to generate the image. (don't forget the -o, otherwise hell will break loose on your PC)
This is what you get:

in my case, the mistake was that I have 2 classes "sudo" in 2 different module

class nesoav2::sudo {
  file { '/etc/sudoers.d/nesoav2':
    source => "puppet:///nesoav2/sudo_nesoav2",
    owner => root,
    group => root,
    mode => 0440,
    require => Class["sudo"]
  }
}

and if you don't specify the other module (package) name in Class["sudo"], the class will try to import itself. This is really a beginner's error in Puppet, like having a class extending itself (try that in Java and the IDE will immediately tell you.... Puppet's IDE is simply dumb).
If you generate the WHOLE graph "dot -Tpng expanded_relationships.dot -o expanded_relationships.png" it's really way too complex. One should have the option of plotting only the nodes related to a specific resource....
These are all the commands:
dot -Tpng expanded_relationships.dot -o expanded_relationships.png
dot -Tpng relationships.dot -o relationships.png
dot -Tpng resources.dot -o resources.png


I read here that a much better way of examining the graphs is to import the .dot file into Gephi

VirtualBox: Running a 64 bit host RHEL OS on a 32 bit Windows machine

My VBOX is a

facter | more

architecture => x86_64
hardwareisa => x86_64
hardwaremodel => x86_64
kernelrelease => 2.6.32-279.el6.x86_64
kernelversion => 2.6.32

processor0 => Intel(R) Core(TM) i5-3437U CPU @ 1.90GHz

so it clearly expects a 64 bit OS and 64 bit Processor.

My CPU is clearly up to the task, but the host OS is clearly not (Windows 7 32 bit)

Contrary to popular belief, it can be made to work, as long as you are allowed to enable VT-D hardware virtualization in your BIOS (most likely you will have to provide a password).

This is documented here http://www.virtualbox.org/manual/ch03.html and here http://www.virtualbox.org/manual/ch10.html#hwvirt.

I have managed to make it work on my laptop, but it has crashed several times with a BLUE SCREEN OF DEATH. This seems to happen whenever I hibernate my computer while the Vistual Machine is up and running.

Once more I really regret having to use a Windows guest OS to work...

All the best to Vinita

She left the project today, she will be missed, but it's very good for her to get more opportunities and advance with her career.

All the best Vinita and keep in touch!

WebLogic: log JMS message body

Unfortunately the WebLogic console missed this option.

See also "Logging the Uniform Distributed Queue Message Body (Doc ID 1377584.1)"

the Oracle DOC gives only the manual workaround (hack! hack!).

with WLST:

moduleName = 'CommonAlertingJMSModule'
queueName = 'CommonAlertingQ'

edit()
startEdit()
cd '/JMSSystemResources/' + moduleName + '/JMSResource/' + moduleName + '/UniformDistributedQueues/' + queueName + '/MessageLoggingParams/' + queueName
cmo.setMessageLoggingFormat('%header%,%properties%,%body%')
save()
activate()

How to return a default value if lookup fails on a table

given this table:

  CREATE TABLE "ACME_ERRORCODES" 
   ( "ERRORCODE" VARCHAR2(100 BYTE), 
 "ISRETRIABLE" NUMBER, 
 "ISCUSTOM" NUMBER
   ) ;

if you want to return a default value 0 for ISRETRIABLE in case the ERRORCODE is missing in the table, you can do:

select NVL ( 
   (select ISRETRIABLE from ACME_ERRORCODES where errorcode = 'SOME_MISSING_ERROR_CODE' ), 
    '0') 
from dual;

WebLogic: how to find JDBCDriver version for oracle.jdbc.OracleDriver

These ways don't use the ACTUAL driver being used by WebLogic.

You can simply run WLST, connect(....) to a Managed Server,

serverRuntime()
cd ('/JDBCServiceRuntime/' + msname + '/JDBCDataSourceRuntimeMBeans/' + dsname)
ls()

and you will see interesting stuff:

-r--   DatabaseProductName                          Oracle
-r--   DatabaseProductVersion                       Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
-r--   DeploymentState                              2
-r--   DriverName                                   Oracle JDBC driver
-r--   DriverVersion                                11.2.0.2.0

PS: interestingly, OracleDriver contains a public static final String getDriverVersion()

Installing puppet-lint on your Vagrant VirtualBox

BEFORE you commit your puppet stuff, you better "lint" it :

http://rubydoc.info/gems/puppet-lint/0.3.2/frames

gem install puppet-lint

Although there are ways to install a gem offline, it's a lot easier to configure a proxy:

export http_proxy=http://myuser:mypassword@myproxyhost:myproxyport
gem install puppet-lint

and then:

cd /tmp/vagrant-puppet/modules-0/osb/manifests
puppet-lint --with-filename instance.pp

instance.pp - ERROR: trailing whitespace found on line 17

OWSM policies: username is not preserved upon import of OSB project in Eclipse

I am not sure this restriction is documented. I have searched in Oracle Support and I could only find this:
Import Policies in OWSM 11g Using a User without Admin Rights (Doc ID 1208863.1)
"Is it possible to import policies in OWSM 11g using a User who does not have Administrative rights?"
"One potential option is to map the logical role "policy.Updater" defined in the WSM-PM EJB application to a group that the User belongs to.
By default, the logical role "policy.Updater" is mapped to the "Administrators" group. "
but this refers to the user you use to log into sbconsole, not to the user you attach to a policy. See also http://docs.oracle.com/cd/E25178_01/web.1111/b32511/managing.htm#CEGHCGEB :
We discovered that Eclipse doesn't import the tag access-control-policies which is created when you edit the oracle/wss11_username_token_with_message_protection_service_policy in the Acces Control tab:

  <ser:coreEntry isProxy="true" isEnabled="true" isTracingEnabled="true">
    <ser:security>
      <con5:inboundWss processWssHeader="false"/>
      <con5:access-control-policies>
        <con5:transport-level-policy xsi:type="con6:ProviderPolicyContainerType" xmlns:con6="http://www.bea.com/wli/sb/security/accesscontrol/config">
          <con6:policy provider-id="XACMLAuthorizer">
            <con6:policy-expression>Usr(pippo)|Usr(pluto)</con6:policy-expression>
          </con6:policy>
        </con5:transport-level-policy>
      </con5:access-control-policies>
    </ser:security>

The only workaround is to open the .proxy file with a text editor and PASTE manually the access-control-policies tag. It's a bit sad.

OSB and OWSM Policy "not supported"

We have some OSB Proxy Services with OWMS policies attached.
When deploying some services, the error message "[OSB Security - OWSM:387177]OWSM Policy oracle/wss_saml_or_username_token_service_policy is not supported " is displayed.
Nothing is displayed in the logs.
Finally it turned out that we need to deploy the OSB project with both Managed Servers up and running.
Thanks Rajesh for solving this issue.

The long struggle with Eclipse Subclipse plugin

I had to change password in SVN, and the new password was not being saved in svn.simple. For every operation I get the username/password popup. Running me crazy.

Googling, I find this article, so I do all the tricks:
deleted .keyring file
deleted the svn.simple file
Windows/Preferences/SVN and changed the Configuration Location to something else
started Eclipse as administrator
looked in Eclipse .log file in workspace looking for errors

Nothing worked. I HATE Eclipse.

Finally switching from SVN Interface JavaHL to SVNKit seems to fix the issue.

Unless you HAVE to use Eclipse, seriously consider switching to IntelliJ. Seriously.

Charting infrastructure with Graphite

Graphite is a metrics collecting and charting engine, it seems really well done (in Python!) and with excellent performance. One can use it with collectd to chart CPU usage, disk, memory etc.

http://graphite.wikidot.com/faq

Even an idiot like can learn how to use its dashboard.

Useful tips to install Firefox on the company's laptop

You are behind a firewall, which occasionally could prevent you from installing Flash Player. No worries: download the whole offline installer here and be happy. However it still requires admin rights, unfortunately.

You don't have Admin rights to install Firefox: don't worry, you can use the Portable edition

You keep getting an annoying popup "This web page is being redirected to a new location.". No worries, just follow instructions here ( about:config, then disable network.http.prompt-temp-redirect ).

Groovy: parse a file line by line, split, sort unique list of words

I know, in bash this would be a one liner... however when things become more complicated, your bash code becomes hell, while Groovy maintains its readability

print "Hello, welcome to the WordParser 1.0\n"

rootDir = "C:\\pierre\\downloads\\istdaseinmensch\\"
myfile = new File(rootDir + "Levi,_Primo_-_Ist_das_ein_Mensch.txt")

myWords = []
countWords = 0
countLines = 0

myfile.eachLine { line ->
 if (line.trim().size() == 0) {
  return null
 } else {
  countLines++
  words = line.split("[^A-Za-z0-9]+")
  for (theWord in words) {
      if (theWord.length() > 0 && !Character.isDigit(theWord.charAt(0))) {
 countWords++
 myWords.add(theWord.toLowerCase())
      }
  }
 }
}

print "countLines=" + countLines + " countWords=" + countWords + "\n"

myUniqueWords = myWords.unique().sort()

print "unique words = " + myUniqueWords.size() + "\n"

new File(rootDir + "out.txt").withWriter { out ->
 myUniqueWords.each {
  out.println(it)
 }
}

Next: how to invoke google translate REST API to get a translation for each word, and produce a readable output where each word has a mouse-over hint displaying its translation.
PS try doing this in Puppet, it will be ready by the end of time and meanwhile most of the functions you have used are no longer supported.

Installing git on Ubuntu

"apt-get install git" not necessarily will work, especially if you are behind a proxy. This is how I have done it:
sudo su - root
export http_proxy=http://myusername:mypassword@myproxyhost:myproxyport
apt-get install git

If you get:

E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

then make sure you are root, and eventually if it keep bugging you, just restart the box. This worked for me.

All these things could be self evident for a Linux administrator, but surely some more helpful error messages would not hurt...

Puppet sucks?

It could suck more. It could also suck less.
After some time using Puppet, these are my thoughts - straight from my heart.
- for someone having any Groovy, Python or Java background, it's EXTREMELY PAINFUL to accept the limitations of the language

- a pathetically poor library of data manipulation functions (dimly enhanced with stdlib-puppet),

- no iterations, no easy way to transform data structures into a different structure

- very scarce OO support (there is class inheritance but it's deprecated (???!!!) ) - copy and paste syndrome is the natural outcome

- no support for nested resources - everything is at a flat level - to achieve nesting you must write classes, which introduces abstraction which IMHO is not always a good thing

- 2 constructs - define and class - whose use cases show a confusingly ambiguous separation

- same class "declaration" construct expressed in 2 totally different ways: "include classname" and "class { 'classname': parametername => parametervalue ...}" . This is like speaking German and Russian in the same sentence, it's just confusing.
- a jargon which openly contradicts well consolidated standards from Software development... like "declare" a class instead of "instantiating a class".... to me, "declare" means "define the class' prototype", and not "instantiate".

- no enumerations (HORRIBLE! Everything is a String!)

- failure upon redefinition of the same resource name (can't you give me a simple option/construct to simply ignore the second declaration?)

- limited support for packaging and context separation between modules

- a DSL which is ugly and verbose (all those : and , .... and then, have you ever seen a list ending with a COMMA ???? It's ugly!)

- for those missing complex data modeling, hash data structures are there, but boy, how painful and fragile it is to use them

- no simple options to hook to a CMDB, you must use those hiera and yaml that to me are brittle and limited. Just give me an option to use a DB, it's my decision, why do you impose me to use YAML or JSON, I hate property files to maintain configuration.

- the seemingly best IDE available, Geppetto, is way more primitive (e.g. no refactoring available - which to me is an ESSENTIAL feature) than even the most basic Java IDE

- lack of a java-like "import modulename.* " statement to avoid having to verbosely specify the fully qualified name of each define/class defined in another module

- lack of notion of a scope/context to keep modules separated (sort of "module" notion in Java)

- Ruby???? How many sysops have ever used Ruby before? I mean, Python is 3 times more popular than Ruby, and probably 10 times more in the sysadmin community....

On the whole, although Puppet is a very good idea, the implementation is simply too limited in its options. I have said the same about Maven. It will evolve with time until it will incorporate all the features of a modern OO language - it's already evolving with 3.2 version - however it would have been much better to do it since the beginning.

On the whole using Puppet one has the sensation that the last 30 years of Software Engineering have passed in vain.

If Infrastructure is CODE (which is a fascinating concept, and I am all for it) then why not have at disposition the BEST coding languages on the market? Why should we castrate ourselves with a very limited DSL? Why reinventing the wheel to make it square?

Andrey says:

Initially they targeted the language towards system administrators, and programmers could always the Ruby API if they want, but now when Puppet got so much attention from the industry they realize that Puppet DSL is way too simple for what it is already used.

I noticed that shops where programmers dominate tend to choose Chef over Puppet, because Chef is a Ruby DSL and you can always use full power of Ruby inside Chef recipes and cookbooks. But organizations where there is a huge weight of system administrators, who have little or no programming experience, tend to choose Puppet. After all, Puppet DSL was never meant to be a programming language, but rather a modeling language (kind of UML for infrastructure) to describe the state desired of the system.

It's not that I'm defending Puppet DSL, the language clearly sux, but I definitely see now some legit reasoning behind initial language design.




On a final note, I would definitely believe that in the long run Puppet will vanish, replaced by something with a more sound design, more solid and refactorable, encompassing more modern paradigms... what will replace Puppet, no clue, maybe it's not even born yet, but surely it's too a huge business not to interest BIG implementors.

Github: Failed to connect to github.com:443; No error

I have been swearing for a while to realize that the git client (http://msysgit.github.com/) requires a proxy settings to be able to contact github.
Here are all the commands (don't tell me git is intuitive.... it's a hell of a lot of work only to get the basic done).
The command is "git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080"
All the git config settings are documented here

$ git config --global user.name "vernetto"

$ git config --global user.email pvernetto@gmail.com

$ mkdir kitammuort

$ cd kitammuort/

$ git init
Initialized empty Git repository in c:/pierre/workspaceGIT/kitammuort/.git/

$ touch README

$ git add README

$ git commit -m 'first commit'
[master (root-commit) e5310e1] first commit
 1 file changed, 1 insertion(+)
 create mode 100644 README

$ git remote add origin https://github.com/vernetto/kitammuort.git

$ git push origin master
fatal: unable to access 'https://github.com/vernetto/kitammuort.git/': Failed to connect to github.com:443; No error

 
git remote set-url origin https://vernetto@github.com/user/kitammuort.git

git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080

git pull origin master

git push origin master

Book: Vagrant Up and Running

I am not a big fan of Vagrant, don't ask me where this antipathy comes from (maybe because I still don't understand why all these operations/infrastructure people use Ruby - a web development language - instead of Groovy, Java or Python).... however it's a valuable tool once you acquire the discipline and you organize your work.

The best of Puppet Conf 2013

You can find all videos here

However some of them are not extremely exciting, so better pick only some.

This one "getting started with Puppet" from Michael Stahnke is absolutely captivating, this guy knows how to talk to an audience.

This one is a bit verbose but contains some useful info on how to write modules and classes: Forging Great Modules



Culturally fascinating the talk by Jez Humble (great guy IMHO), although not very technically focused

Google Infrastructure: several considerations on networking issues, importance of testing rollbacks, have in place a strategy to handle ANY failure (not really directly related to Puppet).

...more to come....

Book: Puppet 3 Beginner's guide

This is an excellent hands-on tutorial to get started with Pupppet. I suggest this should be your first book on Puppet before you move on more advanced topics. The author wrote also the other book "Puppet Cookbook".

OSB: Custom XPath vs Java Callout classloader difference

the Custom Xpath classloader doesn't access the jars in the domain lib folder (like /opt/oracle/domains/osbpl1do/lib folder), while the java callout does. So a function might work when invoked by JavaCallout, but give a nasty error from a customxpath:

 java.lang.ClassNotFoundException: org.apache.commons.codec.binary.Hex
 at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:297)
 at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:270)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:179)

If you need some common libraries in a Custom Xpath, you better add the jar explicitly to the JVM classpath

vi /opt/oracle/domains/osbpl1do/bin/setDomainEnv.sh

EXT_POST_CLASSPATH=/opt/oracle/domains/osbpl1do/lib/commons-codec-1.4.jar

restart your managed server
Check the JVM classpath:

pid="the managed server pid"
jinfo -sysprops ${pid} | grep java.class.path | awk -F'=' '{print $2}'

this gives me this new classpath:

....many more
/opt/oracle/fmw11_1_1_5/wlserver_10.3/server/lib/xqrl.jar
/opt/oracle/domains/osbpl1dom/lib/commons-codec-1.4.jar
/opt/oracle/fmw11_1_1_5/patch_wls1035/profiles/default/sys_manifest_classpath/weblogic_patch.jar
....many more

For more on WebLogic classloader, see here

More on saving BLOB with Oracle DBAdapter

This will NOT work:

CREATE TABLE BLATABLE (BLAFIELD BLOB);

INSERT INTO BLATABLE (BLAFIELD) VALUES ('SOMEVALUE');

you get a ORA-01465 - "invalid hex number"

When I try to save a String to a BLOB field in a stored procedure, I need to convert a String to a bytearray. Using UTL_RAW.CAST_TO_RAW works for strings of < 2K, for longer strings you hit the limit of the internal implementation of this function:

UTL_RAW.CAST_TO_RAW(theString)
ORA-06502: PL/SQL: numeric or value error: raw variable length too long

Using UTL_RAW.TO_RAW entails that you use a hex representation of a string; if you pass any non-hex string you get:

UTL_RAW.TO_RAW(theString) ORA-01465: invalid hex number

You can test that actually UTL_RAW.TO_RAW converts hex to RAW format:
UTL_RAW.TO_RAW('3435') = '45'

http://docs.oracle.com/html/A90373_03/ap_urpkg.htm
In XQuery there is no built-in function to transform String or XML in Hex. Sad.
You can either write yourself such XQuery (good luck), or use Apache Common Hex class in a custom Xpath A good way is this:

public static String fromXMLObjectToHexString(org.apache.xmlbeans.XmlObject theXMLObject) {
 return Hex.encodeHexString(theXMLObject.toString().getBytes());
}

OracleDB: How to save a byte array to a BLOB field in Java

PreparedStatement statement= getConnection().prepareStatement("INSERT INTO MYTABLE(MYBLODFIELD) VALUES(?)");


byte binData[] = myString.getBytes();
inStream = new ByteArrayInputStream(binData)
statement.setBinaryStream(1, inStream, binData.length);
statement.executeUpdate();

How to retrieve a BLOB in Java and display as XML

see also this on WLST

public class DBConn {

    public static String getBlobData(String driver, String url, String username, String password, String sqlQuery) throws Exception {
 String result = "";
 Class.forName(driver);
 Connection conn = DriverManager.getConnection(url, username, password);
 try {
     Statement stmt = conn.createStatement();
     try {
  ResultSet rset = stmt.executeQuery(sqlQuery);
  try {
      rset.next();
      java.sql.Blob blob = (java.sql.Blob) rset.getBlob(1);
      if (blob != null) {
   byte[] bdata = blob.getBytes(1, (int) blob.length());
   String text = new String(bdata);
   text = text.replace("<", "<");
   result = text;
      } else {
   result = "empty body";
      }

  } finally {
      try {
   rset.close();
      } catch (Exception e) {
   e.printStackTrace();
      }
  }
     } finally {
  try {
      stmt.close();
  } catch (Exception e) {
      e.printStackTrace();
  }
     }
 } finally {
     try {
  conn.close();
     } catch (Exception e) {
  e.printStackTrace();
     }
 }
 return result;
    }

}

Book: Pulling string with Puppet

Excellent book by James Turnbull, which explains in clear words the concepts behind Puppet.
Resources and Collections (classes)
Idempotence; RAL (Resource Abstraction Layer) and Providers
Installation of Puppet, certificates, site.pp
Class inheritance and override. Difference between a class and a definition.
Variable scope. if, case, conditional selector.
node definition, default, node inheritance; virtual and exported resources;
filebucket, tidy
reporting, external node classifier (with examples). Facter and Custom facts.