I have gone over the "WebSphere MQ V7.0 Features and Enhancements", the "Migration Guide"....and this was not mentioned once. I did find reference to this in SupportPac MP1G...but it was after the fact and I had already converted to V7.01.
In version 7.01 of MQ for z/OS....IBM changed the way messages are stored on the pageset. If a message is under 2K....it is stored 1 message per page. This was huge for one of my applications.....as they kept filling the pageset, and overflowing to the dead letter queue. It took a little while to figure this out. Regardless of the design implications....this application would get data all day, and then process it at night. Well it didn't take long to fill the pageset up to the 4GB limit. These messages are 153 bytes each.......so we use a 4K page to store 153 bytes. We went from approximately 20 messages per page....to one.
The solution was to allocate a new "extended" VSAM dataset to be able to go over the 4GB limit. Once this was allocated and formatted......I then dynamically added the pageset to the queue manager:
-qmgr DEFINE PSID(9) BUFFPOOL(3) DSN('MQM.qmgr.PSID09') EXPAND(USER)
Then I created a new STORAGE CLASS called EXPANDED to use this pageset...and finally, I had the application process all of the data...stopped all incoming channels so the queue stayed empty, and then reassigned the queue to the new storage class.
Remember when you add a pageset dynamically, you will still need to change the PROC to include the new DD statement for the pageset, as well as to update the CSQINP1 input dataset to include the pageset to buffer pool entry:
DEFINE PSID( 09 ) BUFFPOOL( 3 )
as well as the input for CSQINP2, where you assign the storage class:
DEFINE STGCLASS( 'EXTENDED') +
PSID( 09 )
There is also another work around that is talked about in the MP1G Performance SupportPac...and it is having this statement in your CSQINP2 DD statement:
REC QMGR (TUNE MAXSHORTMSGS 0)
This will have the queue manager store messages in the pre v7.01 way....more than one on a page.
It seems to me that this should have been pointed out in the migration guide, as well as mentioned in the features and enhancements book. Having to search for information on this after the fact, was not really what I had in mind.
Saturday, May 21, 2011
Saturday, April 9, 2011
Don't Ignore your FDC Probes.
I cannot tell you how many client sites I have been at where they seem to have unexplainable behavior in their WebSphere MQ environments. The first question I ask them is "Have you looked at your FDC probes?" Most of the time they do not know what I am talking about. So I explain, that MQ will tell you when it does not like something in the environment, or is having some type of trouble. In some of my engagements, the MQ servers are running fine, at least there is no apparent problem, but when I look....I usually find some area where tuning is needed.
What I am referring to is the FDC files that MQ cuts when it has experienced a failure. On the distributed platforms..ie. Unix, Linux..etc. these are stored in /var/mqm/errors. Most of the time when an FDC event happens, the programs, services and channels will keep running, so it it not readily noticeable that a problem has occurred.
As MQ administrators, one of our functions is to monitor these files, trying to detect a pattern, or at least cataloging and quantifying what has happened. Armed with this information, we can make adjustments to our environments and get them running to full capacity.
So let's look at getting at the probes...and how many of which type occurred. To do this, we will use some simple Unix/Linux commands.
First off, change to the /var/mqm/errors directory:
$ cd /var/mqm/errors
Now, lets get the probe number, and a count of how many times each has happened:
$ grep "Probe Id" *.FDC|cut -c 38-47 | sort | uniq -c
This is just an example, but your output might look something like this:
9 HL047028
4 HL049110
9 HL077070
47 XC307070
43 XC308010
You might have some lines that are offset and duplicates...but that is due to your FDC file name length and this is based on the process ID number. You can add them together for a total occurrence count.
Okay...these "Probe Id's" are coded based on the first two characters as to which MQ component had the failure. In our case, we have HL and XC. The HL is the "Logger" component and the XC is "Common Services". A partial list of these probe codes can be found on my website, http://www.tpmq-experts.com/FDC-Probes.html.
Okay...as we can see, we had 37 XC307070 (Long Lock Wait) probes and 43 XC308010 (Mutex Release) probes. This is not a catastrophic failure....but does indicate that you are using linear logging and are taking media images at a time when your queues are at a high depth. This does affect the speed of processing for your applications, so it would be good to have a check in your record image script to check the queue depth....if it is high, then it would be wise to wait to perform the record image.
Looking at the other probes, we had 9 HL047028 (Logger problem), 4 HL049110 (Logger problem) and 9 HL077070 (disk full). The last probe, is self explanatory...the disc was full. So if you are using Linear logging...you clean-up script needs to be adjusted, or more disk space allocated. The other two probes point at kernel parameters msgmnb, msgtql and msgseg..which can all be adjusted to handle the load that your current environment needs.
These are just a few examples of probes that can be corrected, which will have a positive impact on the running of your environment and the throughput of your applications.
Have a look at your probes...and really see what's going on.
What I am referring to is the FDC files that MQ cuts when it has experienced a failure. On the distributed platforms..ie. Unix, Linux..etc. these are stored in /var/mqm/errors. Most of the time when an FDC event happens, the programs, services and channels will keep running, so it it not readily noticeable that a problem has occurred.
As MQ administrators, one of our functions is to monitor these files, trying to detect a pattern, or at least cataloging and quantifying what has happened. Armed with this information, we can make adjustments to our environments and get them running to full capacity.
So let's look at getting at the probes...and how many of which type occurred. To do this, we will use some simple Unix/Linux commands.
First off, change to the /var/mqm/errors directory:
$ cd /var/mqm/errors
Now, lets get the probe number, and a count of how many times each has happened:
$ grep "Probe Id" *.FDC|cut -c 38-47 | sort | uniq -c
This is just an example, but your output might look something like this:
9 HL047028
4 HL049110
9 HL077070
47 XC307070
43 XC308010
You might have some lines that are offset and duplicates...but that is due to your FDC file name length and this is based on the process ID number. You can add them together for a total occurrence count.
Okay...these "Probe Id's" are coded based on the first two characters as to which MQ component had the failure. In our case, we have HL and XC. The HL is the "Logger" component and the XC is "Common Services". A partial list of these probe codes can be found on my website, http://www.tpmq-experts.com/FDC-Probes.html.
Okay...as we can see, we had 37 XC307070 (Long Lock Wait) probes and 43 XC308010 (Mutex Release) probes. This is not a catastrophic failure....but does indicate that you are using linear logging and are taking media images at a time when your queues are at a high depth. This does affect the speed of processing for your applications, so it would be good to have a check in your record image script to check the queue depth....if it is high, then it would be wise to wait to perform the record image.
Looking at the other probes, we had 9 HL047028 (Logger problem), 4 HL049110 (Logger problem) and 9 HL077070 (disk full). The last probe, is self explanatory...the disc was full. So if you are using Linear logging...you clean-up script needs to be adjusted, or more disk space allocated. The other two probes point at kernel parameters msgmnb, msgtql and msgseg..which can all be adjusted to handle the load that your current environment needs.
These are just a few examples of probes that can be corrected, which will have a positive impact on the running of your environment and the throughput of your applications.
Have a look at your probes...and really see what's going on.
Saturday, March 26, 2011
Sending, reading and receiving a WebSphere MQ message using SQL.
A little understood and exploited feature of WebSphere MQ and DB2, is the ability to send, read and receive a message using a simple SQL statement. The validity of using this feature we will leave for the systems architects and application developers. The purpose of this post is to explain how it works and how to configure the set-up.
During the installation of DB2, job DSNTIJSG creates several user defined functions. Some of these functions are the AMI interface from DB2 to WebSphere MQ.
To see which functions were created, you can issue the following SQL statement in SPUFI or QMF:
SELECT DISTINCT(NAME) FROM SYSIBM.SYSROUTINES WHERE SCHEMA = 'DB2MQ';
You should see the following:
MQREAD
MQREADALL
MQREADALLCLOB
MQREADCLOB
MQRECEIVE
MQRECEIVEALL
MQRECEIVEALLCLOB
MQRECEIVECLOB
MQSEND
In this post we will specifically be dealing with the MQSEND, MQREAD and MQRECEIVE user defined functions.
There are two tables that control the MQ Services within DB2:
SYSIBM.MQSERVICE_TABLE
SYSIBM.MQPOLICY_TABLE
The MQSERVICE_TABLE contains the service name, queue manager name, queue name, CCSID, encoding and a description of the service.
The MQPOLICY_TABLE contains items that are specific to the message itself and its delivery. You can think of the policy table as a mapping of a lot of the fields in the MQ Message Descriptor (MQMD). Items like, priority, persistence, expiry, message ID, correlation ID, COA, COD, reply to queue, reply to queue manager...and so on.
Let's define a service we can test with. Using SPUFI or QMF, execute the following:
INSERT INTO SYSIBM.MQSERVICE_TABLE VALUES('TEST_DB2_SEND','qmgrName','queueName',500,785,
'A TEST SERVICE');
Looking at the values, we have:
TEST_DB2_SEND - The service name
qmgrName - The name of the Queue Manager that you DB2 can access
queueName - The name of a local, remote or alias queue define on that queue manager
500 - Code character set ID (z/OS)
785 - Encoding (z/OS)
A TEST SERVICE - Description of the service
Now we will need a policy to govern the delivery of this message. For our example, we will keep it simple and specify that the message is to be persistent. So using SPUFI or QMF, execute
INSERT INTO SYSIBM.MQPOLICY_TABLE (POLICYNAME, SEND_PERSISTENCE, DESC)
VALUES('TEST_PERSIST_POLICY','Y','TEST POLICY FOR PERSISTENT MESSAGES');
In this statement, we specify the COLUMN names because there are a lot of columns on the table, and we only want these updated. The others will assume a default value. For the default values you can checkout the DB2 Information Center:
http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db29.doc.apsg/db2z_db2mqtables.htm
So now we have a "service" as well as a "policy" defined, but before we can test the MQSEND function, we will need a Workload Manager (WLM) Application Environment established. Your z/OS systems programmer can set this up for you. Here are the values:
Application Environment
Name DB21MQ
Description TEST
Procedure Name DB21MQ Subsystem DB2
Server Limit No Limit
Start Parameters DB2SSN=&IWMSSNM,NUMTCB=20,APPLENV=DB21MQ
In the above example, we have a DB2 subsystem named DB21. Now we need a procedure in a system PROCLIB called DB21MQ..(see procedure name above).
DB21MQ PROC RGN=0M,APPLENV=DB21MQ,DB2SSN=DB21,NUMTCB=20
IEFPROC EXEC PGM=DSNX9WLM,REGION=&RGN,TIME=NOLIMIT,
PARM='&DB2SSN,&NUMTCB,&APPLENV'
STEPLIB DD DISP=SHR,DSN=db2hlq.SDSNEXIT
DD DISP=SHR,DSN=db2hlq.SDSNLOAD
DD DISP=SHR,DSN=mqhlq.SCSQLOAD
DD DISP=SHR,DSN=mqhlq.SCSQAUTH
DD DISP=SHR,DSN=mqhlq.SCSQANLE
DSSPRINT DD SYSOUT=*
SYSTSPRT DD SYSOUT=*
CEEDUMP DD SYSOUT=*
SYSPRINT DD SYSOUT=*
SYSABEND DD DUMMY
The db2hlq and mqhlq need to be replaced with your high level qualifier for your DB2 and MQ libraries. Okay..we should now be set to give this a test.
To test this we will using SPUFI and issuing a SQL statement to demonstrate the execution.
This SQL statement should have put the message, "TEST MESSAGE WITH NEW POLICY" on the queue you specified with the INSERT statement of the SYSIBM.MQSERVICE table.
Now let's define a service to read the queue. Using the same structure as before:
INSERT INTO SYSIBM.MQSERVICE_TABLE VALUES('TEST_DB2_READ','qmgrName','queueName',500,785,'A TEST READ SERVICE');
And now a policy. For this policy, we will just take the default values. You can tailor your policy to control your specific needs, but for this example..the defaults will work:
INSERT INTO SYSIBM.MQPOLICY_TABLE (POLICYNAME, DESC)
VALUES('TEST_READ_POLICY', 'POLICY FOR MQREAD');
We can now read the message from the queue using the following SQL statement:
SELECT DB2MQ.MQREAD('TEST_DB2_READ', 'TEST_READ_POLICY')
FROM SYSIBM.SYSDUMMY1;
Your output should display the contents of the message we placed on the queue. Using the MQREAD function, the message is still on the queue. To remove the message we will use the MQRECEIVE function.
So using the same policy as we did in the MQREAD...we can issue the statement:
SELECT DB2MQ.MQRECEIVE('TEST_DB2_READ', 'TEST_READ_POLICY')
FROM SYSIBM.SYSDUMMY1;
Your output should show the contents of the message, and the message is now removed from the queue.
You will probably want to name your services and policies something that makes sense based on the queue names and the specific behavior you are looking for. But this gives you an idea of what you are capable of doing. One practical use for these features would be to send and receive messages from a JDBC client without having the z/OS WebSphere MQ Client Attach Facility installed.
Keep messaging!!!!!
During the installation of DB2, job DSNTIJSG creates several user defined functions. Some of these functions are the AMI interface from DB2 to WebSphere MQ.
To see which functions were created, you can issue the following SQL statement in SPUFI or QMF:
SELECT DISTINCT(NAME) FROM SYSIBM.SYSROUTINES WHERE SCHEMA = 'DB2MQ';
You should see the following:
MQREAD
MQREADALL
MQREADALLCLOB
MQREADCLOB
MQRECEIVE
MQRECEIVEALL
MQRECEIVEALLCLOB
MQRECEIVECLOB
MQSEND
In this post we will specifically be dealing with the MQSEND, MQREAD and MQRECEIVE user defined functions.
There are two tables that control the MQ Services within DB2:
SYSIBM.MQSERVICE_TABLE
SYSIBM.MQPOLICY_TABLE
The MQSERVICE_TABLE contains the service name, queue manager name, queue name, CCSID, encoding and a description of the service.
The MQPOLICY_TABLE contains items that are specific to the message itself and its delivery. You can think of the policy table as a mapping of a lot of the fields in the MQ Message Descriptor (MQMD). Items like, priority, persistence, expiry, message ID, correlation ID, COA, COD, reply to queue, reply to queue manager...and so on.
Let's define a service we can test with. Using SPUFI or QMF, execute the following:
INSERT INTO SYSIBM.MQSERVICE_TABLE VALUES('TEST_DB2_SEND','qmgrName','queueName',500,785,
'A TEST SERVICE');
Looking at the values, we have:
TEST_DB2_SEND - The service name
qmgrName - The name of the Queue Manager that you DB2 can access
queueName - The name of a local, remote or alias queue define on that queue manager
500 - Code character set ID (z/OS)
785 - Encoding (z/OS)
A TEST SERVICE - Description of the service
Now we will need a policy to govern the delivery of this message. For our example, we will keep it simple and specify that the message is to be persistent. So using SPUFI or QMF, execute
INSERT INTO SYSIBM.MQPOLICY_TABLE (POLICYNAME, SEND_PERSISTENCE, DESC)
VALUES('TEST_PERSIST_POLICY','Y','TEST POLICY FOR PERSISTENT MESSAGES');
In this statement, we specify the COLUMN names because there are a lot of columns on the table, and we only want these updated. The others will assume a default value. For the default values you can checkout the DB2 Information Center:
http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db29.doc.apsg/db2z_db2mqtables.htm
So now we have a "service" as well as a "policy" defined, but before we can test the MQSEND function, we will need a Workload Manager (WLM) Application Environment established. Your z/OS systems programmer can set this up for you. Here are the values:
Application Environment
Name DB21MQ
Description TEST
Procedure Name DB21MQ Subsystem DB2
Server Limit No Limit
Start Parameters DB2SSN=&IWMSSNM,NUMTCB=20,APPLENV=DB21MQ
In the above example, we have a DB2 subsystem named DB21. Now we need a procedure in a system PROCLIB called DB21MQ..(see procedure name above).
DB21MQ PROC RGN=0M,APPLENV=DB21MQ,DB2SSN=DB21,NUMTCB=20
IEFPROC EXEC PGM=DSNX9WLM,REGION=&RGN,TIME=NOLIMIT,
PARM='&DB2SSN,&NUMTCB,&APPLENV'
STEPLIB DD DISP=SHR,DSN=db2hlq.SDSNEXIT
DD DISP=SHR,DSN=db2hlq.SDSNLOAD
DD DISP=SHR,DSN=mqhlq.SCSQLOAD
DD DISP=SHR,DSN=mqhlq.SCSQAUTH
DD DISP=SHR,DSN=mqhlq.SCSQANLE
DSSPRINT DD SYSOUT=*
SYSTSPRT DD SYSOUT=*
CEEDUMP DD SYSOUT=*
SYSPRINT DD SYSOUT=*
SYSABEND DD DUMMY
The db2hlq and mqhlq need to be replaced with your high level qualifier for your DB2 and MQ libraries. Okay..we should now be set to give this a test.
To test this we will using SPUFI and issuing a SQL statement to demonstrate the execution.
SELECT DB2MQ.MQSEND('TEST_DB2_SEND', 'TEST_PERSIST_POLICY',
'TEST MESSAGE WITH NEW POLICY')
FROM SYSIBM.SYSDUMMY1;
This SQL statement should have put the message, "TEST MESSAGE WITH NEW POLICY" on the queue you specified with the INSERT statement of the SYSIBM.MQSERVICE table.
Now let's define a service to read the queue. Using the same structure as before:
INSERT INTO SYSIBM.MQSERVICE_TABLE VALUES('TEST_DB2_READ','qmgrName','queueName',500,785,'A TEST READ SERVICE');
And now a policy. For this policy, we will just take the default values. You can tailor your policy to control your specific needs, but for this example..the defaults will work:
INSERT INTO SYSIBM.MQPOLICY_TABLE (POLICYNAME, DESC)
VALUES('TEST_READ_POLICY', 'POLICY FOR MQREAD');
We can now read the message from the queue using the following SQL statement:
SELECT DB2MQ.MQREAD('TEST_DB2_READ', 'TEST_READ_POLICY')
FROM SYSIBM.SYSDUMMY1;
Your output should display the contents of the message we placed on the queue. Using the MQREAD function, the message is still on the queue. To remove the message we will use the MQRECEIVE function.
So using the same policy as we did in the MQREAD...we can issue the statement:
SELECT DB2MQ.MQRECEIVE('TEST_DB2_READ', 'TEST_READ_POLICY')
FROM SYSIBM.SYSDUMMY1;
Your output should show the contents of the message, and the message is now removed from the queue.
You will probably want to name your services and policies something that makes sense based on the queue names and the specific behavior you are looking for. But this gives you an idea of what you are capable of doing. One practical use for these features would be to send and receive messages from a JDBC client without having the z/OS WebSphere MQ Client Attach Facility installed.
Keep messaging!!!!!
Sunday, February 27, 2011
WebSphere MQ V7 for z/OS and MQ Explorer....
One of the nice offerings in WebSphere MQ V7 on z/OS is the ability
for MQ Explorer to client attach without having to purchase the z/OS
Client Attachment Facility.
In version 7 on z/OS you are now allowed up to 5 client attachments
for use with MQ Explorer. This will allow administrators and enterprise
monitoring groups to view, monitor and alter z/OS Queue Managers using
the nice Windows interface that MQ Explorer offers.
To download the new version of MQ Explorer, go to: http://www01.ibm.com/software/integration/wmq/explorer/
The set-up for this is seemingly simple. On the z/OS side, you will need the
SYSTEM.ADMIN.SVRCONN channel defined. If you have the Client
Attach Facility installed, the MAXINST attribute on this channel can be set
between 0 and 999,999,999, without the Client Attach Facility the value for
MAXINST is limited to 5. The only other MQ object you will need is the SYSTEM.MQEXPLORER.REPLY.MODEL model queue.
Once these are set-up, your connection from MQ Explorer can be established. You will notice in thenew version of MQ Explorer there is a logon area for your z/OS ID and password. If you are the z/OS MQ administrator, you probably already have the necessary access to the queue manager and all of the objects, so no additional RACF (or ACF2, TopSecret) changes will be needed. However, if you are allowing others to use MQ Explorer who do not currently have access to the queue manager and objects, the following will be needed:
- RACF UPDATE authority to the system queue
SYSTEM.MQEXPLORER.REPLY.MODEL
- RACF UPDATE authority to the queues, AMQ.MQEXPLORER.*
- CONNECT authority to the target queue manager
- Correct authority for the action they specify. This can be different,
for different types of users.
- READ authority to all of the hlq.DISPLAY.object profiles in the MQCMDS
class
Enjoy!!
for MQ Explorer to client attach without having to purchase the z/OS
Client Attachment Facility.
In version 7 on z/OS you are now allowed up to 5 client attachments
for use with MQ Explorer. This will allow administrators and enterprise
monitoring groups to view, monitor and alter z/OS Queue Managers using
the nice Windows interface that MQ Explorer offers.
To download the new version of MQ Explorer, go to: http://www01.ibm.com/software/integration/wmq/explorer/
The set-up for this is seemingly simple. On the z/OS side, you will need the
SYSTEM.ADMIN.SVRCONN channel defined. If you have the Client
Attach Facility installed, the MAXINST attribute on this channel can be set
between 0 and 999,999,999, without the Client Attach Facility the value for
MAXINST is limited to 5. The only other MQ object you will need is the SYSTEM.MQEXPLORER.REPLY.MODEL model queue.
Once these are set-up, your connection from MQ Explorer can be established. You will notice in thenew version of MQ Explorer there is a logon area for your z/OS ID and password. If you are the z/OS MQ administrator, you probably already have the necessary access to the queue manager and all of the objects, so no additional RACF (or ACF2, TopSecret) changes will be needed. However, if you are allowing others to use MQ Explorer who do not currently have access to the queue manager and objects, the following will be needed:
- RACF UPDATE authority to the system queue
SYSTEM.MQEXPLORER.REPLY.MODEL
- RACF UPDATE authority to the queues, AMQ.MQEXPLORER.*
- CONNECT authority to the target queue manager
- Correct authority for the action they specify. This can be different,
for different types of users.
- READ authority to all of the hlq.DISPLAY.object profiles in the MQCMDS
class
Enjoy!!
Subscribe to:
Posts (Atom)