User:Alexeina/Workbench

From FFXI Wiki

Migration of Table Data to New Template with NimbleText



Welcome to this minimal guide using NimbleText for migrating NM and Adversaries tables to new templates.
This guide is about the second step of a common IT process called ETL (Extract, Transform, Load). There's a decent article at Wikipedia if you'd like to know more.
There are many tools and methods to do ETL work but I'm using NimbleText which has a low barrier of entry compared to other tooling.
You do not need to buy the software as this guide only uses features available in the free online version and free desktop version.

Tooling

  • NimbleText - Text manipulation and code generation tool
  • Live Version - Runs in the browser; least amount of features
  • Desktop Free Version - Requires installing application; a few more features than the Live Version
  • Desktop Pro Version - Honestly you don't need this; but it has more features a poweruser would appreciate
  • Notepad++ - Text editor with many useful features; but I use the Find and Replace capabilities most of all during these edits


Data Migration Process

A step-by-step walkthrough using NimbleText to migrate data to a new template.


Step 1
On BGwiki edit the page to view the table data
On Clopedia click on the View Source button in the upper right area of the page/article

Examples of BGwiki NM template names

Zone Bestiary Table
Zone NM Table

Examples of BGwiki monster template names

Zone Monster Instance Table
Zone Adversaries Table

Refer to the Pattern Lookup Table to identify the correct NimbleText Pattern you'll need.

Step 2
Refer to the NimbleText Patterns in this guide and use the indicated values for Column separator, Row separator, and NimbleText Pattern. Copy/Paste these values into NimbleText

Step 3
Copy the data you want to convert/migrate into the NimbleText data area (top area of UI)

Step 4
We need to modify the first lines and last lines of the data for the Row separator to work reliably.
Edit the beginning of the data to match the token of the Row separator.

Example using Row separator of }}/n{{ at the beginning of the data

}}
{{The beginning of data for the first NM entry

Example using Row separator of }}/n/n{{ at the end of the data

|The end of the data for the last NM entry
}}

{{


Make the top row (first row) have }} after these edits the top of the data should look similar to this

}}
{{Zone Adversaries Row


Go to the bottom (last row) of the input data Edit the bottom of the data to look similar to this (make sure there is no empty row or newline at the end of the data)

|Adversaries.Steal=
}}
{{



Step 6
Press the Calculate button located underneath the NimbleText pattern area.

Step 7
Copy everything from the results area (bottom area of NimbleText UI)

Step 8
Paste into the wiki editing window and Show preview Compare the preview against original table (can view original table in another browser window)
Refer to the Alzadaal Undersea Ruins zone page for an example of how NM and Adversaries tables should appear

Step 9
Look over the preview and make any manual edits necessary If you don't already have a Text Editor preference I'll recommend Notepad++ for manual editing because there are often many items that need the same edits; using Replace and Find is a real time saver. https://notepad-plus-plus.org/ Keep in mind that the Mob's page should be considered the source of truth, these tables are supposed to be a convenient summary for the zone. Please always make any corrections or additions to mob/nm information mob's page first and then adding that information to these tables is a secondary concern.

Examples of manual edits

  • Fixing drops that are KI; these are not handled properly by the Pattern at the time this guide is written
  • Icon and Image file names are typically shortened on bgwiki thus you'll need to replace those as well
  • Sometimes map positions have extra ( )
  • Spawn Conditions in the input data are found in too many ways for me to anticipate in the migration pattern; Spawn Conditions always require manual editing
  • Family/Genus is sometimes a broken link because the input data was incorrect

NimbleText Pattern Lookup Table

Wiki Site Data Type Origin Template Name NimbleText Pattern
BGwiki NM Zone Bestiary Table BG-NM-1
BGwiki NM Zone NM Table BG-NM-2
BGwiki Monster Zone Monster Instance Table BG-Monster-1
BGwiki Monster Zone Adversaries Table BG-Monster-2
Clopedia NM tableMobLine-NM Clopedia-NM-1
Clopedia Monster tableMobLine-Regular Clopedia-Monster-1

NimbleText Patterns

This section contains all the NimbleText Patterns used to manipulate the input data.

Column separator exists consistently between each field within the table row.
Conceptually the Column is a particular piece of information such as the monster's name, level, and family.

Row separator exists consistently between each table row.
Conceptually the Row contains all the information for a monster contained within a table.

BG-NM-1

Column separator

\n|


Row separator

}}\n{{


NimbleText Pattern BG-NM-1

$ONCE
== Notorious Monsters ==
{{Zone NM Table 2
|Zone NM Row=
$EACH
{{Zone NM Row 2
|NM.Name=<%  $1.split('=')[1].trim()  %>
|NM.Family=<%  $6.split('=')[1].trim()  %>
|NM.Main=
|NM.Sub=
|NM.Aggressive=<% 
if($5.split('=')[1]) 
     { 'Y' }  %>
|NM.Detects=<%  
if($5.split('=')[1].split(',')[0]) 
     { '{{' + $5.split('=')[1].trim().replace(/\s*,\s*/g, '}} {{').replace(/-/g, ' ') + '}}' }  %>
|NM.Level=<%  $2.split('=')[1].trim()  %>
|NM.Spawn_Condition='''TODO''': <%  $7.split('=')[1].trim()  %>
|NM.Treasure=<%    
if($4.split('=')[1]) {
    let str = '';
    for(let i=1; i<20; i++) {
	if($4.split('*')[i]) {
	str += '\n* {{ItemIcon|' + $4.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('*')[i].trim() + '|' +  $4.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + ' description.png|link=}}';
	}
    }
}  %>  
|NM.Steal=
|NM.Position=(<%  $3.split('=')[1].trim()  %>)<br />Map 1
|NM.Map=TODOmap
}}
$ONCE
}}

BG-NM-2

Column separator

\n|


Row separator

}}\n{{


NimbleText Pattern BG-NM-2

$ONCE
== Notorious Monsters ==
{{Zone NM Table 2
|Zone NM Row=
$EACH
{{Zone NM Row 2
|NM.Name=<% $1.split('=')[1].trim() %>
|NM.Family=<% if($2.split('=')[1].split('|')[1]){ $2.split('=')[1].split('|')[0].replace('[[:Category:', '')} else {$2.split('=')[1].replace('[[', '').replace(']]', '') } %>
|NM.Main=<% $3.split('=')[1].trim() %>
|NM.Sub=<% $4.split('=')[1].trim() %>
|NM.Aggressive=<% $6.split('=')[1].trim().replace('N', '').replace('n', '').replace('F', '').replace('f', '').replace('T', 'Y').replace('t', 'Y').replace('A', 'Y') %>
|NM.Detects=<% $7.split('=')[1].trim().replace('}{', '} {') %><% if($5.split('=')[1].replace('N', '').replace('n', '').replace('F', '').replace('f', '').replace('T', 'Y').replace('t', 'Y')) { ' {{Links}}'} %>
|NM.Level=<% $8.split('=')[1].trim() %><% if($9.split('=')[1]) { '-' + $9.split('=')[1].trim() } %>
|NM.Spawns=<% $10.split('=')[1].replace('x', '').trim() %>
|NM.Spawn_Condition='''Timed''': ?min. <% $11.split('=')[1].replace(/\n/g, " ") %>
|NM.Treasure=<%
if($12.split('=')[1]) {
    let str = '';
    for(let i=1; i<20; i++) {
		if($12.split('*')[i]) {
		str += '\n* {{ItemIcon|' + $12.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $12.split('*')[i].trim() + '|' +  $12.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + ' description.png|link=}}';
		}
    }
}
%>
|NM.Steal=<%
if($13.split('=')[1]) {
    let str = '';
    for(let i=0; i<10; i++) {
		if($13.split('=')[1].split(',')[i]) {
			if(i > 0) { str += ','; }
			str += ' {{ItemIcon|' + $13.split('=')[1].split(',')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $13.split('=')[1].split(',')[i].trim() + '|' +  $13.split('=')[1].split(',')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + ' description.png|link=}}';
		}
    }
}
%>
|NM.Position=(<% $14.split('=')[1].replace('((', ')').replace('))', ')').trim() %>)<br />Map
|NM.Map=<% $15.split('=')[1].trim() %>
}}
$ONCE
}}

BG-Monster-1

Column separator

|


Row separator

}}\n{{


NimbleText Pattern BG-Monster-1

$ONCE
== Adversaries ==
{{Zone Adversaries Table 2
|Zone Adversaries Row 2=
$EACH
{{Zone Adversaries Row 2
|Adversaries.Name=$1
|Adversaries.Family=<% if($2.split('|')[1]){ $2.split('|')[0].replace('[[:Category:', '').replace('Bats', 'Flock Bat').replace('Clot', 'Slime') } else {$2.replace('Bats', 'Flock Bat').replace('Clot', 'Slime').replace('Shadow', 'Fomor').replace('Fly Trap', 'Flytrap').replace('Giant Bird', 'Greater Bird') } %>
|Adversaries.Main=$3
|Adversaries.Sub=
|Adversaries.Aggressive=<% $7.replace('Yes', 'Y').replace('No', '').replace('N', '').replace('A', 'Y') %>
|Adversaries.Detects=<% if($6.split(',')[0]) { '{{' + $6.trim().replace(/\s*,\s*/g, '}} {{').replace(/-/g, ' ').replace('Low HP', 'HP').replace('Job Ability', 'JA').replace('+', '}} {{') + '}}' } %><% if($8 == 'Yes'|| $8 == 'L') { ' {{Links}}'} %>
|Adversaries.Minimum_Level=$4
|Adversaries.Maximum_Level=$5
|Adversaries.Spawns=$9
|Adversaries.Spawn_Condition='''Timed''': ?min. $10 $11
|Adversaries.Treasure=
|Adversaries.Steal=
}}
$ONCE
}}

BG-Monster-2

Column separator

\n|


Row separator

}}\n{{


NimbleText Pattern BG-Monster-2

$ONCE
== Adversaries ==
{{Zone Adversaries Table 2
|Zone Adversaries Row 2=
$EACH
{{Zone Adversaries Row 2
|Adversaries.Name=<% $1.split('=')[1].trim() %>
|Adversaries.Family=<% $2.split('=')[1].trim() %>
|Adversaries.Main=<% $3.split('=')[1].trim() %>
|Adversaries.Sub=<% $4.split('=')[1].trim() %>
|Adversaries.Aggressive=<% $6.split('=')[1].trim().replace('No', '').replace('N', '').replace('n', '').replace('F', '').replace('f', '').replace('T', 'Y').replace('t', 'Y').replace('A', 'Y').replace('a', 'Y') %>
|Adversaries.Detects=<% if($7.split('=')[1].split(',')[0]) { '{{' + $7.split('=')[1].trim().replace(/\s*,\s*/g, '}} {{').replace('/', '}} {{').replace(/-/g, ' ').replace('Low HP', 'HP').replace('Job Ability', 'JA').replace('+', '}} {{') + '}}'  } %><% if($5.split('=')[1].replace('No', '').replace('N', '').replace('n', '').replace('F', '').replace('f', '')) { ' {{Links}}'} %>
|Adversaries.Minimum_Level=<% $8.split('=')[1].trim() %>
|Adversaries.Maximum_Level=<% $9.split('=')[1].trim() %>
|Adversaries.Spawns=<% $10.split('=')[1].trim() %>
|Adversaries.Spawn_Condition='''Timed''': ?min. <% $11.split('=')[1] %>
|Adversaries.Treasure=<%
if($12.split('=')[1]) {
    let str = '';
    for(let i=1; i<20; i++) {
		if($12.split('*')[i]) {
		str += '\n* {{ItemIcon|' + $12.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $12.split('*')[i].trim() + '|' +  $12.split('*')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + ' description.png|link=}}';
		}
    }
}
%>
|Adversaries.Steal=<%
if($13.split('=')[1]) {
    let str = '';
    for(let i=0; i<10; i++) {
		if($13.split('=')[1].split(',')[i]) {
			if(i > 0) { str += ','; }
			str += ' {{ItemIcon|' + $13.split('=')[1].split(',')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $13.split('=')[1].split(',')[i].trim() + '|' +  $13.split('=')[1].split(',')[i].split('|')[0].trim().replace('[[', '').replace(']]', '') + ' description.png|link=}}';
		}
    }
}
%>
}}
$ONCE
}}

Clopedia-NM-1

Column separator

| 


Row separator

}}\n\n{{


NimbleText Pattern Clopedia-NM-1

$ONCE
== Notorious Monsters ==
{{Zone NM Table 2
|Zone NM Row=
$EACH
{{Zone NM Row 2
|NM.$1
|NM.$5
|NM.Main=
|NM.Sub=
|NM.Aggressive=
|NM.Detects=$9
|NM.Level=$3
|NM.Spawn_Condition=
|NM.Treasure=
* {{imgpop|[[TODO]]|TODOTreasureNameHere description.png|link=}}
|NM.Steal=
|NM.Position=$2
|NM.Map=TODO MAP FILE HERE
}}
$ONCE
}}

Clopedia-Monster-1

Column separator

| 


Row separator

}}\n\n{{


NimbleText Pattern Clopedia-Monster-1

$ONCE
== Adversaries ==
{{Zone Adversaries Table 2
|Zone Adversaries Row 2=
$EACH
{{Zone Adversaries Row 2
|Adversaries.Name=<% $1.split('=')[1].trim() %>
|Adversaries.Family=<% if($6.split('=')[1].trim().split('').pop() == 's') { $6.split('=')[1].trim().slice(0, -1) } else { $6.split('=')[1].trim() } %>
|Adversaries.Main=
|Adversaries.Sub=
|Adversaries.Aggressive=<% if($8.split('=')[1].split(',')[0].trim() == 'A') {'Y';} %>
|Adversaries.Detects=<% $8.split('=')[1].trim().replace('}}', '').replace('NA', '').replace('A', '').replace('L', '{{Links}}').replace('S', '{{Sight}}').replace('H', '{{Sound}}').replace('HP', '{{HP}}').replace('M', '{{Magic}}').replace('Sc', '{{Scent}}').replace('T(S)', '{{True Sight}}').replace('T(H)', '{{True Sound}}').replace('JA', '{{JA}}').replace('WS', '').replace('Z(D)', '').replace('Z(N)', '').replace('A(R)', '').replace(',', '').replace(',', '').replace(',', '').replace(',', '').replace(',', '').replace(',', '').replace(',', '').replace(',', '').replace('}{', '} {').replace('}{', '} {').trim() %>
|Adversaries.Minimum_Level=<% if($3.split('=')[1].split('-')[0]) { $3.split('=')[1].split('-')[0].trim()  } %>
|Adversaries.Maximum_Level=<% if($3.split('=')[1].split('-')[1]) { $3.split('=')[1].split('-')[1].trim()  } %>
|Adversaries.Spawns=<% $7.split('=')[1].trim() %>
|Adversaries.Spawn_Condition='''Timed''': ?min. <% $2.split('=')[1] %>
|Adversaries.Treasure=<% if($4.split('=')[1].split('<br>')[0]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[0].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[0].trim() + '|' +  $4.split('=')[1].split('<br>')[0].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' } %><% if($4.split('=')[1].split('<br>')[1]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[1].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[1].trim() + '|' +  $4.split('=')[1].split('<br>')[1].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[2]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[2].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[2].trim() + '|' +  $4.split('=')[1].split('<br>')[2].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[3]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[3].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[3].trim() + '|' +  $4.split('=')[1].split('<br>')[3].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[4]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[4].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[4].trim() + '|' +  $4.split('=')[1].split('<br>')[4].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[5]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[5].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[5].trim() + '|' +  $4.split('=')[1].split('<br>')[5].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[6]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[6].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[6].trim() + '|' +  $4.split('=')[1].split('<br>')[6].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[7]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[7].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[7].trim() + '|' +  $4.split('=')[1].split('<br>')[7].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[8]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[8].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[8].trim() + '|' +  $4.split('=')[1].split('<br>')[8].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }   %><% if($4.split('=')[1].split('<br>')[9]) { '\n* {{ItemIcon|' + $4.split('=')[1].split('<br>')[9].split('|')[0].trim().replace('[[', '').replace(']]', '') + '|22}} {{imgpop|' +  $4.split('=')[1].split('<br>')[9].trim() + '|' +  $4.split('=')[1].split('<br>')[9].split('|')[0].trim().replace('[[', '').replace(']]', '') +  ' description.png|link=}}' }  %>
|Adversaries.Steal=<% if($5.split('=')[1].split('<br>')[1]) { '{{ItemIcon|' + $5.split('=')[1].split('<br>')[0].replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + '|22}} {{imgpop|' + $5.split('=')[1].split('<br>')[0].replace('([[Despoil]])', '').trim() + '|' + $5.split('=')[1].split('<br>')[0].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + ' description.png|link=}}' } %><% if($5.split('=')[1].split('<br>')[1]) { '\n{{ItemIcon|' + $5.split('=')[1].split('<br>')[1].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + '|22}} {{imgpop|' + $5.split('=')[1].split('<br>')[1].replace('([[Despoil]])', '').trim() + '|' + $5.split('=')[1].split('<br>')[1].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + ' description.png|link=}}' } %><% if($5.split('=')[1].split('<br>')[2]) { '\n{{ItemIcon|' + $5.split('=')[1].split('<br>')[2].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + '|22}} {{imgpop|' + $5.split('=')[1].split('<br>')[2].replace('([[Despoil]])', '').trim() + '|' + $5.split('=')[1].split('<br>')[2].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + ' description.png|link=}}' } %><% if($5.split('=')[1].split('<br>')[3]) { '\n{{ItemIcon|' + $5.split('=')[1].split('<br>')[3].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + '|22}} {{imgpop|' + $5.split('=')[1].split('<br>')[3].replace('([[Despoil]])', '').trim() + '|' + $5.split('=')[1].split('<br>')[3].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + ' description.png|link=}}' } %><% if($5.split('=')[1].split('<br>')[4]) { '\n{{ItemIcon|' + $5.split('=')[1].split('<br>')[4].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + '|22}} {{imgpop|' + $5.split('=')[1].split('<br>')[4].replace('([[Despoil]])', '').trim() + '|' + $5.split('=')[1].split('<br>')[4].trim().replace('[[', '').replace(']]', '').replace('([[Despoil]])', '').trim() + ' description.png|link=}}' } %>
}}
$ONCE
}}

Javascript References

NimbleText Patterns support in-line Javascript which I've used quite a bit.
<% and %> indicate the beginning and end of Javascript code in the Patterns.
If you want to modify the Patterns the below Javascript references may be of help.

NimbleText Help

Contact me on the Blue Gartr discord with any questions, but please keep your expectations low as I'm an amateur at using NimbleText and Javascript programming.

References

Online Help Documentation

Tips

  • Javascript loops - disable Auto Calculate, when creating loops an infinite loop will make NimbleText unresponsive

Limitations

  • Javascript loops - loops inside the Pattern will only output data from the last iteration. Work around this limitation by concatenating the result of your string manipulations at the end of every loop iteration.