RHQ

Support for paging in the RHQ REST-api (updated)

May 23, 2013 6:05 AM by Heiko Rupp

I have just pushed some changes to RHQ master that implement paging for (most) of the REST-endpoints that return collections of data.



If you remember, I was asking the other day if there is a "standard" way to do this. Basically there are two
"big options":

1. Put paging info in the Http-Header
2. Put paging info in the body of the message

While I think paging information are meta data that should not "pollute" the body, I understand the arguments from the JavaScript side that says that they don't easily have access to the http headers
within AJAX requests. So what I have now done is to implement both:

1. Paging in the http header: this is the standard way that you get if you just request the "normal" media types of application/xml or application/json (output from running RestAssured):
   
   [update]
   My colleague Libor pointed out that the links do not match with format from RFC 5988 Web Linking, which is now fixed.
   [/update]

   
   Request method:	GET
   Request path:	http://localhost:7080/rest/resource
   Query params:	page=1
   				ps=2
   				category=service
   Headers:		Content-Type=*/*
   				Accept=application/json
   HTTP/1.1 200 OK
   Server=Apache-Coyote/1.1
   Pragma=No-cache
   Cache-Control=no-cache
   Expires=Thu, 01 Jan 1970 01:00:00 CET
   Link=<http://localhost:7080/rest/resource?ps=2&category=service&page=2>; rel="next"
   Link=<http://localhost:7080/rest/resource?ps=2&category=service&page=0>; rel="prev"
   Link=<http://localhost:7080/rest/resource?ps=2&category=service&page=152>; rel="last"
   Link=<http://localhost:7080/rest/resource?page=1&ps=2&category=service>; rel="current"
   Content-Encoding=gzip
   X-collection-size=305
   Content-Type=application/json
   Transfer-Encoding=chunked
   Date=Thu, 23 May 2013 07:57:38 GMT
   
   [
       {
           "resourceName": "/",
           "resourceId": 10041,
           "typeName": "File System",
           "typeId": 10013,
   …..
   

2. Paging as part of the body - there the "real collection" is wrapped inside an object that also contains paging meta data as well as the paging links. To request this representation, a media type of application/vnd.rhq.wrapped+json needs to be used (and this is only available with JSON at the moment):
   

   
   Request method:	GET
   Request path:	http://localhost:7080/rest/resource
   Query params:	page=1
   				ps=2
   				category=service
   Headers:		Content-Type=*/*
   				Accept=application/vnd.rhq.wrapped+json
   Cookies:		
   Body:			
   HTTP/1.1 200 OK
   Server=Apache-Coyote/1.1
   Pragma=No-cache
   Cache-Control=no-cache
   Expires=Thu, 01 Jan 1970 01:00:00 CET
   Content-Encoding=gzip
   Content-Type=application/vnd.rhq.wrapped+json
   Transfer-Encoding=chunked
   Date=Thu, 23 May 2013 07:57:40 GMT
   
   {
       "data": [
           {
               "resourceName": "/",
               "resourceId": 10041,
               "typeName": "File System",
               "typeId": 10013,
   …
       ],
       "pageSize": 2,
       "currentPage": 1,
       "lastPage": 152,
       "totalSize": 305,
       "links": [
           {
               "next": {
                   "href": "http://localhost:7080/rest/resource?ps=2&category=service&page=2"
               }
           },
   …
   }
   

Please try this if you can. We want to get that into a "finished" state (as for the whole REST-Api) for RHQ 4.8

Creating a delegating login module (for JBoss EAP 6.1 )

May 16, 2013 6:07 PM by Heiko Rupp

[ If you only want to see code, just scroll down ]

Motivation

In RHQ we had a need for a security domain that can be used to secure the REST-api and its web-app via container managed security. In the past I had just used the classical DatabaseServerLoginModule to authenticate against the database.

Now does RHQ also allow to have users in LDAP directories, which were not covered by above module. I had two options to start with:

• Copy the LDAP login modules into the security domain for REST
• Use the security domain for the REST-api that is already used for UI and CLI

The latter option was of course favorable to prevent code duplication, so I went that route. And failed.

I failed because RHQ was on startup dropping and re-creating the security domain and the server was detecting this and complaining that the security domain referenced from the rhq-rest.war was all of a sudden gone.

So next try: don't re-create the domain on startup and only add/remove the ldap-login modules (I am talking about modules, because we actually have two that we need).

This also did not work as expected:

• The underlying AS sometimes went into reload needed mode and did not apply the changes
• When the ldap modules were removed, the principals from them were still cached
• Flushing the cache did not work and the server went into reload-needed mode

So what I did now is to implement a login module for the rest-security-domain that just delegates to another one for authentication and then adds roles on success.

This way the rhq-rest.war has a fixed reference to that rest-security-domain and the other security domain could just be handled as before.

Implementation

Let's start with the snippet from the standalone.xml file describing the security domain and parametrizing the module

 <security-domain name="RHQRESTSecurityDomain" cache-type="default">
   <authentication>
     <login-module code="org.rhq.enterprise.server.core.jaas.DelegatingLoginModule" flag="sufficient">
       <module-option name="delegateTo" value="RHQUserSecurityDomain"/>
       <module-option name="roles" value="rest-user"/>
     </login-module>
   </authentication>
 </security-domain>

So this definition sets up a security domain RHQRESTSecurityDomain which uses the DelegatingLoginModule that I will describe in a moment. There are two parameters passed:

• delegateTo: name of the other domain to authenticate the user
• roles: a comma separated list of modules to add to the principal (and which are needed in the security-constraint section of web.xml

For the code I don't show the full listing; you can find it in git.

To make our lives easier we don't implement all functionality on our own, but extend
the already existing UsernamePasswordLoginModule and only override
certain methods.

public class DelegatingLoginModule extends UsernamePasswordLoginModule {

First we initialize the module with the passed options and create a new LoginContext with
the domain we delegate to:

    @Override
    public void initialize(Subject subject, CallbackHandler callbackHandler, 
        Map<String, ?> sharedState,
        Map<String, ?> options) 
    {
        super.initialize(subject, callbackHandler, sharedState, options);

        /* This is the login context (=security domain) we want to delegate to */
        String delegateTo = (String) options.get("delegateTo");

		/* Now create the context for later use */
        try {
            loginContext = new LoginContext(delegateTo, new DelegateCallbackHandler());
        } catch (LoginException e) {
            log.warn("Initialize failed : " + e.getMessage());
        }

The interesting part is the login() method where we get the username / password and store it for later, then we try to log into the delegate domain and if that succeeded we tell super that we had success, so that it can do its magic.

    @Override
    public boolean login() throws LoginException {
        try {
            // Get the username / password the user entred and save if for later use
            usernamePassword = super.getUsernameAndPassword();
            
            // Try to log in via the delegate
            loginContext.login();

            // login was success, so we can continue
            identity = createIdentity(usernamePassword[0]);
            useFirstPass=true;

            // This next flag is important. Without it the principal will not be
            // propagated
            loginOk = true;

the loginOk flag is needed here so that the superclass will call LoginModule.commit() and pick up the principal along with the roles.
Not setting this to true will result in a successful login() but no principal
is attached.

            if (debugEnabled) {
                log.debug("Login ok for " + usernamePassword[0]);
            }

            return true;
        } catch (Exception e) {
            if (debugEnabled) {
                LOG.debug("Login failed for : " + usernamePassword[0] + ": " + e.getMessage());
            }
            loginOk = false;
            return false;
        }
    }

After success, super will call into the next two methods to obtain the principal and its roles:

    @Override
    protected Principal getIdentity() {
        return identity;
    }


    @Override
    protected Group[] getRoleSets() throws LoginException {

        SimpleGroup roles = new SimpleGroup("Roles");

        for (String role : rolesList ) {
            roles.addMember( new SimplePrincipal(role));
        }
        Group[] roleSets = { roles };
        return roleSets;
    }

And now the last part is the Callback handler that the other domain that we delegate to will use to obtain the credentials from us. It is the classical JAAS login callback handler. One thing that first totally confused me was that this handler was called several times during login and I thought it is buggy. But in fact the number of times it is called corresponds to the number of login modules configured in the RHQUserSecurityDomain.

    private class DelegateCallbackHandler implements CallbackHandler {
        @Override
        public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {

            for (Callback cb : callbacks) {
                if (cb instanceof NameCallback) {
                    NameCallback nc = (NameCallback) cb;
                    nc.setName(usernamePassword[0]);
                }
                else if (cb instanceof PasswordCallback) {
                    PasswordCallback pc = (PasswordCallback) cb;
                    pc.setPassword(usernamePassword[1].toCharArray());
                }
                else {
                    throw new UnsupportedCallbackException(cb,"Callback " + cb + " not supported");
                }
            }
        }
    }

Again, the full code is available in the RHQ git repository.

Debugging (in EAP 6.1 alpha or later )

If you write such a login module and it does not work, you want to debug it. Started with the usual means to find out that my login() method was working as expected, but login just failed. I added print statements etc to find out that the getRoleSets() method was never called. But still everything looked ok. I did some googling and found this good wiki page. It is possible to tell a web app to do audit logging

<jboss-web>
    <context-root>rest</context-root>
    <security-domain>RHQRESTSecurityDomain</security-domain><disable-audit>false</disable-audit>

This flag alone is not enough, as you also need an appropriate logger set up, which is explained on
the wiki page. After enabling this, I saw entries like

16:33:33,918 TRACE [org.jboss.security.audit] (http-/0.0.0.0:7080-1) [Failure]Source=org.jboss.as.web.security.JBossWebRealm;
principal=null;request=[/rest:….

So it became obvious that the login module did not set the principal. Looking at the code in the superclasses then led me to the loginOk flag mentioned above.

Now with everything correctly set up the autit log looks like

22:48:16,889 TRACE [org.jboss.security.audit] (http-/0.0.0.0:7080-1)
[Success]Source=org.jboss.as.web.security.JBossWebRealm;Step=hasRole;
principal=GenericPrincipal[rhqadmin(rest-user,)];
request=[/rest:cookies=null:headers=authorization=user-agent=curl/7.29.0,host=localhost:7080,accept=*/*,][parameters=][attributes=];

So here you see that the principal rhqadmin has logged in and got the role rest-user assigned, which is the one matching in the security-constraint element in web.xml.

Further viewing

I've presented the above as a Hangout on Air. Unfortunately G+ muted me from time to time when I was typing while explaining :-(

After the video was done I got a few more questions that at the end made me rethink the startup phase for the case that the user has a previous version of RHQ installed with LDAP enabled. In this case, the installer will still install the initial DB-only RHQUserSecurityDomain and then in the startup bean
we check if a) LDAP is enabled in system settings and b) if the login-modules are actually present.
If a) matches and they are not present we install them.

This Bugzilla entry also contains more information about this whole story.

Creating Https Connection Without javax.net.ssl.trustStore Property

May 8, 2013 3:38 PM by John Mazz

Question: How can you use the simple Java API call java.net.URL.openConnection() to obtain a secure HTTP connection without having to set or use the global system property "javax.net.ssl.trustStore"? How can you make a secure HTTP connection and not even need a truststore?

I will show you how you can do both below.

First, some background. Java has a basic API to make a simple HTTP connection to any URL via URL.openConnection(). If your URL uses the "http" protocol, it is very simple to use this to make basic HTTP connections.

Problems creep in when you want a secure connection over SSL (via the "https" protocol). You can still use that API - URL.openConnection() will return a HttpsURLConnection if the URL uses the https protocol - however, you must ensure your JVM can find and access your truststore in order to authenticate the remote server's certificate.

[note: I won't discuss how you get your trusted certificates and how you put them in your truststore - I'll assume you know, or can find out, how to do this.]

You tell your JVM where your truststore is by setting the system property "javax.net.ssl.trustStore" and you tell your JVM how to access your truststore by giving your JVM the password via the system property "javax.net.ssl.trustStorePassword".

The problem is these are global settings (you often see instructions telling you to set these values via the -D command line arguments when starting your Java process) so everything running in your JVM must use that truststore. And you can't alter those system properties during runtime and expect those changes to take effect. Once you ask the JVM to make a secure connection, those system property values appear to be cached in the JVM and are used thereafter for the life of the JVM (I don't know exactly where in the JRE code these values are cached, but my experience shows me that they are). Changing those system properties later on in the lifetime of the JVM has no effect; the original values are forever used.

Another problem that some people run into is having the need for a truststore in the first place. Sometimes you don't have a requirement to authenticate the server endpoint; however, you would still like to send your data encrypted over the wire. You can't do this readily since the connection you obtain from URL.openConnection() will, by default, expect to use your truststore located at the path pointed to by the system property javax.net.ssl.trustStore.

To allow me to use different truststores for different connections, or to allow me to encrypt a connection but not authenticate the endpoint, I wrote a Java utility object that allows you to do just this.

The main constructor is this:

public SecureConnector(String secureSocketProtocol,
                       File   truststoreFile,
                       String truststorePassword,
                       String truststoreType,
                       String truststoreAlgorithm)

You pass it a secure socket protocol (such as "TLS") and your truststore file location. If the truststore file is null, the SecureConnector object will assume you do not want to authenticate the remote server endpoint and you only want to encrypt your over-the-wire traffic. If you do provide a truststore file, you need to provide its password, its type (e.g. "JKS"), and its algorithm (e.g. "SunX509") - if you pass in null for type and/or algorithm, the JVM defaults are used.

Once you create the object, just obtain a secure connection to any URL via a call to SecureConnector.openSecureConnection(URL). This expects your URL to have a protocol of "https". If successful, an HttpsURLConnection object is returned and you can use it like any other connection object. You do not need to set javax.net.ssl.trustStore (or any other javax.net.ssl system property) and, as explained above, you don't even need to provide a truststore at all (assuming you don't need to do any authentication).

The code for this is found inside of RHQ's agent - you can read its javadoc and look through SecureConnector code here.

The core code is found in openSecureConnection and looks like this, I'll break it down:

First, it simply obtains the HTTPS connection object from the URL itself:

HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();

Then it prepares a custom SSLContext object using the given secure socket protocol:

TrustManager[] trustManagers;
SSLContext sslContext = SSLContext.getInstance(getSecureSocketProtocol());

If no truststore file was provided, it will build its own "no-op" trust manager and "no-op" hostname verifier. What these "no-op" objects will do is always accept all certificates and hostnames thus they will always allow the SSL communications to flow. This is how the authentication is by-passed:

if (getTruststoreFile() == null) {
    // configured to not care about authenticating server, encrypt but don't worry about certificates
    trustManagers = new TrustManager[] { NO_OP_TRUST_MANAGER };
    connection.setHostnameVerifier(NO_OP_HOSTNAME_VERIFIER);

If a truststore file was provided, then it will be loaded in memory and stored in a KeyStore instance:

} else {
    // need to configure SSL connection with truststore so we can authenticate the server.
    // First, create a KeyStore, but load it with our truststore entries.
    KeyStore keyStore = KeyStore.getInstance(getTruststoreType());
    keyStore.load(new FileInputStream(getTruststoreFile()), getTruststorePassword().toCharArray());

The truststore file's content (now stored in a KeyStore object) is used to initialize a trust manager. Unlike the "no-op" trust manager that was created above (if a truststore file was not provided), this trust manager really does perform authentication and it uses the provided truststore's certificates to authorize the server being communicated with. This is why we no longer need to worry about the system properties "javax.net.ssl.trustStore" and "javax.net.ssl.trustStorePassword" - this builds its own trust manager using the data provided by the caller:

    // create truststore manager and initialize it with KeyStore we created with all truststore entries
    TrustManagerFactory tmf = TrustManagerFactory.getInstance(getTruststoreAlgorithm());
    tmf.init(keyStore);
    trustManagers = tmf.getTrustManagers();
}

Finally, the SSL context is initialized with the trust manager that was created earlier (either the "no-op" trust manager, or the trust manager that was initialized with the truststore's certificates). That SSL context is handed off to the SSL connection so the connection can use the context when it needs to perform authentication:

sslContext.init(null, trustManagers, null);
connection.setSSLSocketFactory(sslContext.getSocketFactory());

The connection is finally returned to the caller, fully configured and ready to be used.

return connection;

This is helpful for certain use cases. First, it is helpful when you have multiple truststores that you need to choose from when connecting to different servers as well as being able to switch truststores at runtime (remember, the system property values of javax.net.ssl.trustStore, et. al. are fixed for the lifetime of the JVM - this helps bypass that restriction). This is also helpful in local testing, debugging and demo scenarios when you don't really need or care about setting up truststores and certificates but you do want to connect over https.

RHQ 4.7 released

May 7, 2013 5:26 AM by Heiko Rupp

RHQ 4.7 has been released and one of the two major features in this release are
the new charts that finally have replaced the year old charts that we had since the
start of RHQ project:

The new charts are implemented on with the awesome D3.js toolkit as I've written before.

The other big change is an upgrade of the underlying app server to JBoss EAP 6.1 alpha 1.

As always there have been many more smaller improvements and bug fixes.

Please check the full release notes for details. They also contain a list of commits.

RHQ is an extensible tool to monitory your infrastructure of machines and applications, alert operators on user defined conditions, configure resources and run operations on them from a central web-based UI. Other ways of communicating with RHQ include a command line interface and a REST-api.

You can download the release from source forge.

As mentioned above, the old installer is gone, so make sure to read
the wiki document describing how to use the new installer.

Maven artifacts are available from the JBoss Nexus repository and should also be available from Central.



Please give us feedback, be it in Bugzilla, Mailing lists or the forum. Or just join us on IRC at irc.freenode.net/#rhq.

Deleting RHQ Agent Made Easier

Mar 14, 2013 4:11 AM by John Mazz

In the past, if you wanted to remove an RHQ Agent from your RHQ environment, the simple answer was "just uninventory the platform" which would, under the covers, also remove the agent record completely.

However, in some cases, users found it difficult to remove their agent. Usually, what happens is they try to install the RHQ Agent, run into problems, and then get their RHQ system in a state that causes their RHQ Agent to not be able to register with the RHQ Server. For example, this could happen when a person runs the agent as a different user from before or with the -L command line option - both of which essentially purges the agent's security token and could cause the RHQ Server to reject future registration requests from the agent.

If a person does not understand the linkage between platform and agent, or if the agent's platform was never committed to inventory, it became difficult to understand how to get out of the quadmire.

This has now been addressed as an enhancement as requested by BZ 849711. Now, the answer is simple - regardless of whether the platform is in inventory or not, and even if the agent's resources do not yet show up in the discovery queue - you have a way to quickly purge your agent from the system. This will allow you to get back to a clean slate and attempt to re-install your agent.

You do this by going to the top Adminstration page and selecting the "Agents" item. From here, you see the list of all the agents currently registered in your RHQ environment. If you select one or more of them, you now have the option of pressing the new "Delete" button at the bottom. This will do a few things. First, if the agent's platform is already in inventory, it will uninventory that platform. This means that platform and all its child servers and services will be removed (so be careful and make sure you really want to do this - you will lose all manageability and all audit history for all resources previously managed by that agent). Once that is done, all resources will disappear from the inventory and you won't even see any resources for that agent in the Discovery Queue. Finally, the agent's record itself is removed - so the Administration>Agents page will show that the agents have disappeared.

 
With the agent and its resources completely removed, you have the option to attempt to re-install the agent if you wish to bring it back.

You can also use this feature if your managed infrastructure has changed and you no longer want to manage a machine. Just select the agent that was responsible for managing that machine and delete it.

Note that if your agent is still running, it will attempt to re-register itself! So if you no longer wish to manage a machine, make sure you shutdown the agent as well (you'll obviously want to do this anyway, since you won't want an RHQ Agent consuming resources on a machine that you no longer want managed by RHQ).

View more rhq