Integrating Shotgun and JIRA

UPDATE: A few months after writing this blog post I discovered that using Webhooks in JIRA is much easier. No plugin is required, just a target URL. I strongly recommend that you look at this instead.


I normally don’t write much about programming and software integration on this blog, but I’ve built up some experience with Shotgun and JIRA at work and I thought I’d share some of my findings.

It is expected that you have some experience with both Shotgun and JIRA, including the use of the Shotgun Event Framework and the ScriptRunner add-on for JIRA. You also need the JIRA and Shotgun libraries for Python.

A reoccurring request among Shotgun and JIRA users is a way to integrate the two programs to automatically update certain fields across one another whenever they’re updated. I’ve figured out a way to do this. It may not be the most elegant way of doing it, and I also hear that the producers of the software products are fiddling with their own solution which I’m sure will be better.

But until then, this solution might just work for you.

Shotgun to JIRA

This part is actually relatively easy as it merely uses the event daemon in the Shotgun Event Framework. I run the Windows service / daemon with the following Python plugin script added to it:

"""
eventAnimChanged.py: Change to 'FPS' field -> JIRA

This is a shotgunEventDaemon plugin.

The 'FPS' field in a Shotgun entity was changed. If the relevant entity has a
valid 'JIRA ID' key, the field is compared to the corresponding field in that
JIRA issue. If there are discrepancies, the update is written to the issue.
"""

import sys, logging
from jira import JIRA

# Remember to adapt these constants to match your own Shotgun and JIRA configuration
SG_API_CODE             = 'your_shotgun_script_API_code'
SG_ENTITY_PAGE          = 'your_shotgun_entity_page'    # E.g. 'CustomEntity16'
FIELD_JIRA_FPS          = 'your_JIRA_field'             # E.g. 'customfield_11201'
JIRA_KEY                = 'start_of_your_JIRA_key'      # E.g. 'abc-'
SITE_JIRA               = 'your_JIRA_site'
USER_JIRA               = {'username': 'a_JIRA_username', 'password': 'a_JIRA_password'}

def registerCallbacks(reg):
    """Register all necessary or appropriate callbacks for this plugin."""
    
    reg.logger.info('---------------- Starting eventAnimChanged.py:')

    eventFilter = {'Shotgun_'+SG_ENTITY_PAGE+'_Change': ['sg_fps']} # Just the 'FPS' field
    reg.registerCallback('eventAnimChanged', SG_API_CODE, eventAnimChanged, eventFilter)

    reg.logger.setLevel(logging.INFO) # Set to e.g. ERROR for less log noise

def eventAnimChanged(sg, logger, event, args):
    """Evaluate if changes to the entity requires the corresponding JIRA issue to be updated."""

    logger.info('---- Event: %s', event)
    logger.info('Affected entity ID %s is "%s"', event['entity']['id'], event['entity']['name'])

    # Fetch specific fields of the entity
    filters = [['id', 'is', event['entity']['id']]]
    fields = sg.find_one(SG_ENTITY_PAGE, filters, ['sg_fps', 'sg_jira_id'])
    logger.info('Fields of interest: %s', fields)

    if not JIRA_KEY in fields['sg_jira_id'].lower():
        logger.info('The entity does not appear to have a valid JIRA key; aborting')
        return

    try:
        jira = JIRA(server=SITE_JIRA, basic_auth=(USER_JIRA['username'], USER_JIRA['password']))
        logger.info(jira)
    except Exception, e:
        logger.error(e)

    # Get corresponding JIRA fields
    issue = jira.issue(fields['sg_jira_id'])
    logger.info('Summary of JIRA issue: "%s"', issue.fields.summary)

    fps = getattr(issue.fields, FIELD_JIRA_FPS)

    # Compare field changes
    if int(fields['sg_fps']) == int(fps):
        logger.info("The contents of the JIRA 'FPS' field is the same; skipping")
    else:
        logger.info("The contents of the JIRA 'FPS' field differs; updating")

    # Update 'FPS' field in JIRA to match the recent change in Shotgun
    issue.update(fields={FIELD_JIRA_FPS: fields['sg_fps']})
    logger.info("Updated JIRA field '%s' to: %s", FIELD_JIRA_FPS, fields['sg_fps'])

The script catches when a user edits the FPS field in the entity page in Shotgun, then modifies the corresponding field in JIRA. The entity page has a field called JIRA ID with the JIRA key. If the JIRA ID field is empty, the script aborts.

JIRA to Shotgun

Time for the interesting part. The way I made it work requires the following:

  • The add-on ScriptRunner for JIRA for using a listener script written in Groovy
  • Administrator access to JIRA to access the Groovy script section
  • Direct access to the in-house JIRA server machine, e.g. via Remote Desktop

The direct access was required to catch an event via a Groovy script attached to the listener event system in JIRA and then pass it on to a Python script on the same machine. As far as I researched there is no such thing as a Shotgun library for Groovy, that’s why I quickly pass on control to a Python script that does. I guess I could have done this by passing on a web argument to a different machine and thus not had to gain access to the JIRA server, but luck had it that I was a JIRA administrator prior to switching over to Shotgun so I did have access to that server. So the local Python method was the one I decided upon.

The add-on was something we already had installed for other Groovy scripts that automated various parts of a project that was later moved to Shotgun. It’s not free, but it’s worth it. It also adds a lot of new JQL commands, scripted fields, etc.

On the JIRA server, the Groovy scripts (related to Java) are running in E:\JIRA\scripts. I created a new listeners folder and added the following script named shotgun_issue_anim.groovy inside of it:

/**
 * SCRIPT LISTENER: Changed Issue in ABC -> Shotgun
 *
 * When an issue in ABC is changed, send the JIRA key of it to a Python script
 * which in turn will evaluate what needs to be updated in Shotgun. This script
 * only needs the key since it can read from JIRA itself.
 *
 * An entity in Shotgun should have a 'JIRA ID' field with the key for this to
 * work. If the key is not found in Shotgun, nothing will happen.
 */

package listeners.shotgun_issue_anim // Use this in the 'Name of groovy class' field

import com.atlassian.jira.event.issue.AbstractIssueEventListener
import com.atlassian.jira.event.issue.IssueEvent
import org.apache.log4j.Category

class ShotgunIssueAnim extends AbstractIssueEventListener {
    Category log = Category.getInstance(ShotgunIssueAnim.class)

    @Override
    void workflowEvent(IssueEvent event) {
        log.setLevel(org.apache.log4j.Level.DEBUG) // Log the event in the latest "atlassian-jira.log" file
        log.debug "Event: ${event.getEventTypeId()} fired for ${event.issue} and caught by ShotgunIssueAnim"

        def script = "c:\\shotgun\\python\\python.exe c:\\shotgun\\scripts\\groovy\\groovyIssueAnim.py "+event.issue.key
        script.execute() // Run the Python script "groovyIssueAnim.py"
    }
}

To make the script work in the JIRA event system, I go to the Add-ons administrator section in JIRA, under SCRIPT RUNNER I click Script Listeners, and here I click the Custom listener link. I then add the listener script (actually a class) like this:

Setting Up a Custom Listener in JIRA

As soon as it’s added, the script (class) is listening to changes in JIRA.

The above script is actually very simple. It catches the JIRA key that did an event and then I immediately call a Python script with this key as the argument. Since this takes place on the JIRA server itself, I merely call this script in C:\Shotgun\scripts\groovy:

"""
groovyIssueAnim.py: Evaluate JIRA issue in ABC that was just changed.

Use: Called by a Groovy script through the ScriptRunner plugin in JIRA.

An argument with the JIRA key is received. The JIRA issue is read and compared
to the entity in Shotgun that has the same key in a 'JIRA ID' field. If there
are discrepancies, the updates are written to similar fields in the Shotgun
entity. If the JIRA key is not found in Shotgun, nothing will happen.
"""

import sys, logging
from jira import JIRA

from shotgun_api3 import Shotgun

# Remember to adapt these constants to match your own Shotgun and JIRA configuration
SITE_SG                 = 'your_shotgun_web_address'
SG_API_CODE             = 'your_shotgun_script_API_code'
SG_PROJECT_ID           = your_shotgun_project_id       # E.g. 42
SG_ENTITY_PAGE          = 'your_shotgun_entity_page'    # E.g. 'CustomEntity16'
FIELD_JIRA_FPS          = 'your_custom_JIRA_field'      # E.g. 'customfield_11201'
JIRA_KEY                = 'start_of_your_JIRA_key'      # E.g. 'abc-'
SITE_JIRA               = 'your_JIRA_site'
USER_JIRA               = {'username': 'a_JIRA_username', 'password': 'a_JIRA_password'}
PATH_LOG                = 'folder_location_of_log_file'

if __name__ == '__main__':

    # Start logging
    logging.basicConfig(filename = PATH_LOG+'\\'+os.path.basename(sys.argv[0])[:-3]+'.log', level = logging.INFO, format='%(asctime)s %(levelname)s: %(message)s')
    logging.info('---------------- Starting '+os.path.basename(sys.argv[0])+':')

    if not JIRA_KEY in sys.argv[1].lower():
        logging.error('Argument "%s" does not appear to be a valid JIRA key', sys.argv[1])
        quit()
    else:
        logging.info('JIRA key received in argument: %s', sys.argv[1])

    try:
        jira = JIRA(server=SITE_JIRA, basic_auth=(USER_JIRA['username'], USER_JIRA['password']))
        logging.info(jira)
    except Exception, e:
        logging.error(e)

    # Get JIRA fields
    issue = jira.issue(sys.argv[1])
    logging.info('Summary of JIRA issue: "%s"', issue.fields.summary)
    
    fps = getattr(issue.fields, FIELD_JIRA_FPS)
    logging.info('FPS field from JIRA reads: %s', int(fps))
    
    # Get permission to make changes in Shotgun
    sg = Shotgun(SITE_SG, 'groovyIssueAnim', SG_API_CODE)
    logging.info('Shotgun object instantiated: %s', sg)

    # Shotgun: Find the entity that contains the same key in its 'JIRA ID' field
    filters = [
        ['project', 'is', {'type': 'Project', 'id': SG_PROJECT_ID],
        ['sg_jira_id', 'is', sys.argv[1]]
    ]
    fields = sg.find_one(SG_ENTITY_PAGE, filters, ['id', 'code', 'sg_fps'])
    logging.info('Entity in Shotgun with the same JIRA key is "%s"', fields['code'])

    # Compare field changes
    if int(fields['sg_fps']) == int(fps):
        logging.info("The contents of the Shotgun 'FPS' field is the same; skipping")
    else:
        logging.info("The contents of the Shotgun 'FPS' field differs; updating")

        # Update 'FPS' field in Shotgun to match the recent change in JIRA
        data = {
            'sg_fps': int(fps)
        }
        result = sg.update(SG_ENTITY_PAGE, fields['id'], data)
        logging.info("Updated 'FPS' field in Shotgun entity: %s", result)

The Python script gets the key, reads the relevant JIRA issue, then reads the entity in Shotgun that has the same key in its JIRA ID field. It compares, and if different, updates Shotgun with the value from JIRA.

By the way, the Groovy script logs to atlassian-jira.log on the server. You can find the pertinent debug lines by searching for “AjaxIssueAction” in it.

We tested the change of the FPS field forth and back using the above method. It seems to take a few seconds for the value to make it. We tried machine gun editing a field to see how the other side handled it (I can’t remember which way). It actually missed out on a few in between, but it did receive the last one which was the most important. I hope (and believe) it’s not a problem in production.

Please ask in the comments if you have any questions about this.

2 comments on “Integrating Shotgun and JIRA

  1. Quick question: This appears to be a scripting solution for server instances of JIRA. Does it also work with cloud-based instances?

    Fantastic work, by the way. Thank you for sharing it.

  2. Yes, we’re running an in-house server version of JIRA on a virtual machine.

    I’m not familiar with the cloud-based version of JIRA but I doubt this solution would work there since the Groovy script calls a Python script on the server itself. That would require access to the local folder system and I can’t imagine that the cloud service would allow that.

    That being said, I suppose the Groovy script could then be rewritten to call a script on another domain address instead of directly on the server itself. That might work.

Leave a Reply