You don’t have to master queueing theory to get results

A lot has been said about the importance of “off-lining jobs,” which is essentially deferring heavy, time and resource consuming transactions to the background. When using offline processing, users don’t have to wait for a transaction to complete while accessing your website or application. We see this commonly in credit card transactions, email creation, and PDF generation. Additionally, offline processing helps you with the ability to better control the load and utilization of resources at peak load times.

Off-lining activities usually means a background system for managing queues is required as part of the setup. It can be based on the basic operating system scheduler (like cron, a time-based job scheduler), self-developed systems, or commercial ones (like BMC’s Control-M or Zend Server Job Queue).

An example of a very simple queue system

An example of a very simple queue system

Job creation and system configuration

Taking selected parts of the code for asynchronous execution as an offline job is a worthwhile, reasonable effort conducted by technical code changes. Once the desired parts are defined as jobs, it is up to the job system to handle the queuing and execution. Job systems can usually be configured to manage several queues, several job servers, and priorities.

Zend Server Job Queue can be configured to handle jobs and queues priorities, scheduling and reoccurrence, execution concurrency, managing retries and failures, and more.

Let’s assume you’ve chosen your system and created the jobs as suggested. And now you ask yourself:

  • Are you done with it?
  • How do I configure my Job Queue system?
  • Is this optimized for my needs?
  • How many jobs failed and why?
  • How long are the jobs waiting before they can run?

Tough questions, hard to answer. Most systems come with default settings, but usually the system basic configuration is not good enough – no one size fits all.

Do you have to master queueing theory to optimize your job queue system and get results? Definitely not.

Optimize offline jobs using Zend Server Job Queue Pulse

Zend Server 9.1 introduces Job Queue Pulse, which collects the analytical information and insights obtained by monitoring the entire Job Queue system and visually presents them.

The Job Queue Panel page in Zend Server console.[/

The Job Queue Panel page in Zend Server console.[/

Job Queue Pulse displays high-level statistics such as the number of jobs that ran, average number of jobs per minute, and average wait time per job. This information provides clarity on how loaded your job system is. In cases of high wait times, you can increase the number of queues or the concurrency, thus increasing the number of jobs that run in parallel. On the other hand, if the wait times are very low, you may consider reducing the concurrency level, to reduce the load on the backend systems (e.g. email server).

Job Queue Pulse displays the success rate and detailed information about failures. Clicking the graphs drills down to a detailed list of jobs, so you can continue analyzing which jobs failed and why. In that context, Pulse is integrated with the Zend Server monitoring system, where it collects the most critical events that are related to the jobs, so you can decide what to handle and solve first. Obviously, you can filter the view per period, application, or queue.

To conclude, by using Job Queue Pulse, you can better optimize, analyze, and fine-tune asynchronous and scheduled Jobs processing. The result is better resources utilization and improved end-user customer experience.

Learn more about what’s new in Zend Server 9.1. Download and try Zend Server 9.1.

The following two tabs change content below.

    Mickey Hoter

    Mickey Hoter is a senior product manager at Zend, a Rogue Wave company. He has 15 years of experience in building web-based products - as a developer, team lead, group manager and product manager. Mickey holds a B.Sc. in Information Systems and an MBA, both from the Technion –the Israeli Institute of Technology. He is passionate about productivity, usability and digital experience, which is why he chose to work for Zend. Outside of the office, Mickey enjoys spending his time with his four children and puts a lot of effort in educating them to love Mother Nature, where most of his hobbies take place.

    About Mickey Hoter

    Mickey Hoter is a senior product manager at Zend, a Rogue Wave company. He has 15 years of experience in building web-based products - as a developer, team lead, group manager and product manager. Mickey holds a B.Sc. in Information Systems and an MBA, both from the Technion –the Israeli Institute of Technology. He is passionate about productivity, usability and digital experience, which is why he chose to work for Zend. Outside of the office, Mickey enjoys spending his time with his four children and puts a lot of effort in educating them to love Mother Nature, where most of his hobbies take place.