Module com.blackhillsoftware.json
Package com.blackhillsoftware.smf2json.cli

Class Smf2JsonCLI

java.lang.Object
com.blackhillsoftware.smf2json.cli.Smf2JsonCLI
public final class Smf2JsonCLI extends Object
Provide a command line framework for reading SMF data and writing JSON.

This class implements common processing for programs producing JSON from SMF data. This class will:

  • Read SMF data from a file or z/OS DD name
  • Write JSON to a file, z/OS DD name or to stdout
  • Process command line options to specify the input and output

The interface Smf2JsonCLI.Client defines a class to control how SMF data should be converted to JSON.

How to Implement

Implementation requires a main class to start the program, and a class implementing Smf2JsonCLI.Client to process the records.

Example: 

 import java.io.*;
 import java.util.*;
 import com.blackhillsoftware.smf2json.cli.*;
 import com.blackhillsoftware.json.*;
 import com.blackhillsoftware.smf.*;
 import com.blackhillsoftware.smf.smf30.*;
 
 public class Smf2JsonCLISample {
 
     public static void main(String[] args) throws IOException {  
         Smf2JsonCLI.create()
             .includeRecords(30, 5)
             .start(new CliClient(), args);
     }
 
     private static class CliClient implements Smf2JsonCLI.Client {
     
         @Override
         public List<?> processRecord(SmfRecord record) {
             Smf30Record r30 = Smf30Record.from(record);
             if (r30.completionSection() != null) {
                 CompositeEntry completionInfo = new CompositeEntry()
                         .add("time", r30.smfDateTime())
                         .add(r30.identificationSection())
                         .add(r30.completionSection());
                 return Collections.singletonList(completionInfo);
             } else {
                 return Collections.emptyList();
             }
         }
 
         @Override
         public List<?> onEndOfData() {
             return null;
         }
     }
 }

Command Line Usage:

This class sets up and processes the following command line options: 
 usage: class-name [options] <input-name> ...]
 Generate JSON from SMF records
 Options:
     --end <arg>     end SMF record time, exclusive e.g.
                     2023-02-09T21:47:51
  -h,--help          print this message and exit
     --indd <arg>    input DD name
     --out <arg>     output file name
     --outdd <arg>   output DD name
     --parallel      use multiple threads
     --pretty        pretty print json
     --start <arg>   start SMF record time, inclusive e.g.
                     2023-02-09T21:47:51
 
 <input-name> : File(s) containing SMF records. Binary data, RECFM=U or
 V[B] including RDW.
 Specify <input-name> or --indd

Parallel Processing

The --parallel command line option will use multiple threads to generate JSON.

Parallel processing can significantly reduce elapsed time on systems with multiple processors available e.g. typical desktop PCs.

Calls to the user supplied processRecord function are serialized by default. This function will only be called from one thread at a time so you do not need to consider thread safety.

If the processRecord function does significant work e.g. CICS records need to be uncompressed, further improvements in elapsed time can be had by calling processRecord in parallel. Set processRecordIsThreadsafe to true and processRecord can also be parallelized. Obviously, make sure that your processRecord function is actually thread safe before setting this option.

Parallel Processing on z/OS

Parallel processing may be less useful on z/OS because z/OS systems typically have fewer CPUs with spare capacity. Java parallel processing is likely to default to a number of tasks based on the number of zIIP and CP processors on the system, which is probably more than you want.

To control the number of threads, set the JVM option: 

 -Djava.util.concurrent.ForkJoinPool.common.parallelism=<n>
Recommendations:
  • Set -Djava.util.concurrent.ForkJoinPool.common.parallelism to the number of zIIPs available (or the number of logical processors if running in SMT mode). Reduce based on zIIP busy% to avoid having multiple threads waiting for dispatch.
  • Make sure that WLM goals are set appropriately so that this work does not impact more important work.

  • Field Details

  • Method Details

    • create

      public static Smf2JsonCLI create()
      Create a new instance of Smf2JsonCLI
      Returns:
      a new Smf2JsonCLI instance

    • description

      public Smf2JsonCLI description(String description)
      Set the description used in the usage statement header
      Parameters:
      description - the description used in the usage statement header
      Returns:
      this Smf2JsonCLI to allow method chaining

    • includeRecords

      public Smf2JsonCLI includeRecords(int smfType)
      Include a specific record type. If no record types are specifically included, all records will be processed. This method can be called multiple times to include multiple record types.
      Parameters:
      smfType - The SMF record type e.g. 30
      Returns:
      this Smf2JsonCLI to allow method chaining

    • includeRecords

      public Smf2JsonCLI includeRecords(int smfType, int subType)
      Include a specific record type/subtype. If no record types are specifically included, all records will be processed. This method can be called multiple times to include multiple record type/subtypes.
      Parameters:
      smfType - The SMF record type e.g. 30
      subType - The SMF record subtype e.g. 5
      Returns:
      this Smf2JsonCLI to allow method chaining

    • easySmfGsonBuilder

      public EasySmfGsonBuilder easySmfGsonBuilder()
      Get the EasySmfGsonBuilder used to create the Gson instance. This allows you to specify additional EasySmfGsonBuilder options, or additional Gson options using EasySmfGsonBuilder.getGsonBuilder()
      Returns:
      the EasySmfGsonBuilder

    • options

      public Options options()
      Get the Apache Commons CLI Options that will be used to parse the command line. You can add options and then check them using commandLine(String[]).
      Returns:
      the Options

    • commandLine

      public CommandLine commandLine(String[] args)
      Get the Apache Commons CLI CommandLine parsed from the command line arguments. Use this if you need to check the CommandLine arguments received, e.g. if you added options using options();
      Parameters:
      args - the command line args received by the main program
      Returns:
      the CommandLine parsed from the args

    • processRecordIsThreadSafe

      public Smf2JsonCLI processRecordIsThreadSafe(boolean isThreadSafe)
      Specify that the supplied Smf2JsonCLI.Client.processRecord(SmfRecord) function is thread safe. If true, the function can be called simultaneously from multiple threads when parallel processing is selected.

      By default, calls to Smf2JsonCLI.Client.processRecord(SmfRecord) are serialized so that it can only be called from one thread at a time even in parallel mode.

      Normally generating JSON is the most CPU intensive process, so there is not much benefit from parallelizing the processRecord call. However, some records require more time consuming processing, e.g. CICS records which may need to be uncompressed. In that case it can reduce elapsed time to allow records to be processed in parallel.

      The caller is responsible for ensuring that the Smf2JsonCLI.Client.processRecord(SmfRecord) function is in fact thread safe

      Parameters:
      isThreadSafe - true if the function is thread safe
      Returns:
      this Smf2JsonCLI to allow method chaining

    • start

      public void start(Smf2JsonCLI.Client client, String[] args) throws IOException
      Start processing SMF records using the specified client and command line arguments.
      Parameters:
      client - an instance of a class implementing Smf2JsonCLI.Client
      args - the command line args received by the main program. Must be the same arguments passed to commandLine if commandLine was used.
      Throws:
      IOException - if an I/O error occurs reading or writing data