Using Python to Restore JIRA Custom Field Values from Issue History

Consider the following scenario: you move a bunch of issues from one type to another, or from one project to another. You don’t pay much attention and just confirm everything. Only then you find out some custom fields are gone and the data is lost – since the field context does not include the new issue type or project.

Don’t panic! no need to rush to restore the entire instance database from backup! The data is still visible in the issue’s History tab. All you need to do is fix the context of the fields and then restore it from the issue history. In this post I share some Python code (using python-jira module) to help you achieve that.

Step 1: Fix the field’s context

This better be done manually and carefully, if your policy is to maintain strict field configuration (which in general is a good idea since it improves Jira performance).

Go to administration > issues > custom fields and for each “lost” field which you wish to restore, click the cog icon > ‘Configure’ and then ‘Edit Configuration’. Set the proper issue types and projects.

Step 2: Prepare the restoration script

Now let’s prepare a python script to get a single issue key and collects the lost data from the issue history

Step 2a – Connect to JIRA and obtain the issue

# Connect to JIRA
from jira.client import JIRA
jira = JIRA({'server': jira_server_url}, basic_auth=(username, password)

# create mapping for custom field names and types
name_map = dict()
type_map = dict()
for field in jira.fields():
    if field['custom']:
        fname = get_current_field_name(field['name'])
        name_map[fname] = field['id']
        type_map[fname] = field['schema']['type']

# get the issue object - note the 'changelog' setting
issue = jira.issue(issue_key, expand='changelog')

Step 2b – Look for the lost fields in the history and get their original values

# get the values of the lost fields from the history entry where they got nullified
lost_fields = ['field 1', 'field 2']
original_value = dict()
for history in issue.changelog.histories:
    for item in history.items:
        field_name = item.field
        if field_name in lost_fields and not item.toString:
            # get the previous field value - in two forms, see below
            field_value_str = getattr(item, "fromString")  # string value
            field_value = getattr(item, "from")  # raw value (empty for string-based fields)
            field_type = type_map[field_name]
            if field_type == 'datetime':  # need to add milliseconds
                original_value[field_name] = field_value.replace('+', '.000+')
            elif field_type == 'user':  # username
                original_value[field_name] = field_value
            elif field_type == 'group':  # appears as array but we need it as string
                original_value[field_name] = field_value_str.replace('[', '').replace(']', '')
            elif field_type == 'array' and field_value:  # convert to python array - not using "eval" to avoid weird/malicious strings.
                original_value[field_name] = field_value_str.split(',')
                original_value[field_name] = field_value_str

Now debug/test the script on different issues to make sure it can handle the various fields and values you have. If you encounter an exception – most likely the field configuration is not correct, so double-check what you did in step 1.

Step 3 – Add code to restore the collected data

Go over the list of values we collected and update the issue with them. We need to update differently based on the field items, Jira’s REST API is anything but unified… some useful examples can be found here.

Note – we do not update fields which are not empty (e.g. someone re-entered them manually).

fields_to_update = dict()
for field_name in original_value.keys():
    field_real_name = name_map[field_name]
    field_type = type_map[field_name]
    # don't update a field which now has a value
    curr_value = getattr(issue.fields, field_real_name)
    if curr_value is None:
        value = values[field_name]
        if field_type == 'number':
            fields_to_update[field_real_name] = int(value)
        elif field_type in ['string', 'datetime']:
            fields_to_update[field_real_name] = value
        elif field_type in ['user', 'group', 'version']:
            fields_to_update[field_real_name] = {'name': value}
        elif field_type in ['array']:
            # for multi-version pickers, need to use [{'name': x} for x in value] 
            fields_to_update[field_real_name] = [{'value': x} for x in value]
            fields_to_update[field_real_name] = {'value': value}
        # useful debug print if needed. Helps comparing the outcome against the official REST API.
        # print("Adding {}({},{}) -> {}".format(field_name, field_real_name, field_type, fields_to_update[field_real_name]))
# now update the issue
if fields_to_update:
        print("Updating issue " + issue.key)
    except Exception as e:
        print("{}: FAILED: {}".format(issue.key, e))
    print("{}: Nothing to update".format(issue.key))

That’s it! Run it on the affected issues and everything’s back to normal.

Tip – if you’re going to run this script on a bunch of issues, I highly recommend to disable email notifications in your Jira instance beforehand ūüôā

JIRA Pre-Upgrade check – call for votes

Let’s start with a bottom line: if you’re a JIRA user, please take a few seconds to vote for the ‘JIRA Pre-Upgrade check’ suggestion

Now for the whole story:

Recently we upgraded one of our JIRA instances. It all seems well at first and then small problems started to pile up.

After contacting Atlassian support, it turns out we should have upgraded our database server (Oracle in this case) before the upgrade (“it’s in the documentation!”). We didn’t, so some data upgrade tasks silently failed, without any notification during or after the upgrade process.

I was surprised. As a long-time installation developer I learned one thing: users don’t read the documentation until they have problems. They certainly don’t read installation guides; at most they skim through¬†the release notes. That’s why every installation I create includes some basic validations regarding the environment (e.g. OS, database, Java…).

So, I suggested Atlassian to add this kind of checks to their own installer and they immediately agreed. All that’s left is to gather enough votes so it would go into their suggestion review process. Appreciate your help!

JIRA Tip: issues showing on Kanban board but missing on Scrum board

Let’s say you have two JIRA projects, project K using a Kanban board and project S using a Scrum board. Each project with its own set of statuses and workflows.

Now, users of project S want to see issues from project K on their Scrum board. They update the board filter accordingly, but to their surprise, many project K issues are not  visible in the Backlog.

Why? because a JIRA board (either Scrum or Kanban) only shows issues whose statuses are mapped to the board’s columns (e.g. To Do, In Progress, Done).
Issues with unmapped status are not displayed on the board at all. This is useful in some scenarios but confusing in others, such as in the example above.

To remedy the situation, go to the Scrum board’s configuration, and under the ‘Columns’ sections you can map or unmap statuses as you see fit.

Atlassian DevOps Event Impressions

Today I attended an event, organized by ManageWare, titled “Bridging the Gap Between Dev & Ops”, focusing on Atlassian’s tools suite. JIRA is a very popular tool here in Israel, so the main theme of the event was to showcase how productivity and collaboration can be increased when integrating JIRA with other tools.

There were quite a few presentation (detailed at the end of the post) on various DevOps-oriented subjects, such as CI/CD pipelines, monitoring and reporting. Nothing revolutionary, but it was nice to see how a good integration between various tools (not just Atlassian ones…) can really make things flow faster and help focus on generating value for the business.

I only wish some of the sessions were longer, there was a lot more to see; unfortunately, due to time constraints, most presenters had to rush through their materials.

And finally we had some fresh Pizza and ice-cold beer ūüôā

The presentations:

Continue reading

JIRA SOAP API is being deprecated

In the past I mentioned that I¬†use the JIRA::Client Perl module quite frequently. This module is based on JIRA’s SOAP API.

Recently it came to my attention that Atlassian deprecated their SOAP API in JIRA 6.0 and will be removing it altogether in JIRA 7.0 (see here). Anyone who is still using the SOAP API is encouraged to migrate to the REST API.

Specifically for the JIRA::Client module, an alternate module, JIRA::REST, is now available, providing similar functionality.

So if you’re using any SOAP-based access to JIRA, it’s time to move on…

JIRA Behaviour plugin: handling sub-tasks

I was asked¬†to show a warning message when editing an issue custom field, in case there are subtasks to the issue. Since I¬†already had a validator script which was used for a similar purpose, I figured I’ll just copy the code into the relevant behavior entry and it would work. Clean and simple.

Boy, was I wrong.

As you may know, you need the issue object in order to work with its¬†subtasks. In¬†all Script Runner environments (validators/post functions/etc.) you have the readily available ‘issue’ variable. However it does not exist in the Behavior plugin environment. Since both were written by the same developer, I assumed it must be¬†there somewhere, so tried all sorts of stuff¬†to try and¬†find it, but to¬†no avail (why doesn’t it exist in the FieldBehaviours class?).

“Fine”, I said, “let’s get the issue key and obtain the object from it”. Not very elegant, but simple enough.

Wrong again. getFieldById(“issuekey”) returns null. I honestly have no idea why. Again I had to spend a lot of time¬†searching for a reason, then gave up and started looking for a way to work around it. Eventually I¬†figured out the correct way to get the issue object.


import com.atlassian.jira.ComponentManager
FormField field = getFieldByName("MyCustomField")
def issueid = getFieldById("id").value
def issue = ComponentManager.getInstance().getIssueManager().getIssueObject(Long.parseLong(issueid));
Collection subTasks = issue.getSubTaskObjects()
if (!subTasks.empty) {
 field.setHelpText("<div class=\"warningBox\">Attention: the issue has subtasks.</div>")
else {


Getting a JIRA issue’s priority using Perl

JIRA::Client is a great and useful module. Too bad some of its usage is so cryptic!

For example – suppose you want to print an issue’s priority field. if you simply use $issue->{priority} you will get the priority id, not the display name which everyone is used too.
So, how to get the display name? documentation is very vague on this subject. Google doesn’t help much here.

After a lot of trial & error I figured out a way.

Continue reading

Adding Clickable UNC paths to JIRA

I was tasked with the mission to find a way to add a UNC link to a certain shared document, in a way that every user will be able to access it.

The easiest way I found, so far, is to add a default, read-only field to each issue, so whenever creating or working on an issue, the link is available.

Yes, it’s an overkill, but other options such as adding it to the menu bar seem to require too much research and development (and perhaps being affected by the next JIRA update).

Turns out adding a clickable UNC field is a bit tricky… since there is no such field type (UNC link), and writing UNC paths in text fields doesn’t make them clickable, i figured out the following eay:

  • Add a URL field
  • Set the default value to the document path, for example: file://servername/folder/subfolder/filename.doc
  • Make it read-only using the behaviors plug-in.
  • Use a bulk action to update the field for the relevant issues (since setting a default only affects newly created issues).

Simple, effective, and future proof (and yes, database space wasting… but it’s a price I’m happy to pay).


UPDATE: It works fine in IE 11, but¬†doesn’t work in recent Chrome and FireFox versions due to their default security settings. See this KB article for more information.

Migrating to JIRA – Technical

When migrating from ClearQuest to JIRA, there are some several technical pitfalls which one may encounter. In this post I will detail the main issues which I tackled during the migration process.

The general strategy is to use as much of the built-in JIRA fields as possible, since many abilities and plug-ins uses them by default (such as JIRA Agile). It required effort, but it certainly paid off.

Let’s discuss the details, then:

Continue reading