Jgutierrez6:

Arthor Documentation for Future Developer

2025-05-06T20:50:23Z

Jgutierrez6: /* Active Arthor Instances */

== Introduction ==
[https://www.nextmovesoftware.com/downloads/arthor/documentation/Arthor.pdf Here is the link to Arthor's manual]
* Username: ucsf@nextmovesoftware.com
* Password: <Ask jjiteam@googlegroups.com>

Arthor configurations and the frontend files are consolidated in '''/nfs/soft2/arthor_configs/'''.

'''/nfs/soft2/arthor_configs/start_arthor_script.sh''' can start/restart Arthor instances on respective machines.

Launch the script to see the options available.

== How To Download Arthor ==
# Ssh to nfs-soft2 and become root. Prepare directory
#: <source> mkdir /export/soft2/arthor_configs/arthor-<version> && cd /export/soft2/arthor_configs/arthor-<version> </source>
# [https://www.nextmovesoftware.com/downloads/arthor/ Download Software with this link]
#* Username: ucsf@nextmovesoftware.com
#* Password: <Ask jjiteam@googlegroups.com>
# Go to releases. Look for ''' smallworld-java-<version>.tar.gz ''' and copy the link address.
# Download using wget
#: <source> wget --user ucsf@nextmovesoftware.com --password <Ask jjiteam@googlegroups.com> <link address> </source>
# Decompress the file
#* <source> tar -xvf <file_name> </source>

== How To Launch Arthor For The First Time ==
=== Prepare Files and Directories ===
# Ssh to nfs-exc and become root
# Open a port in the firewall
#: <source>firewall-cmd --permanent --add-port=<port_number>/tcp
firewall-cmd --reload
</source>
# Go to Arthor Config directory
#: <source>cd /export/soft2/arthor_configs/arthor-<latest_version></source>
# Create an Arthor config file
#: <source>vim <name_of_file>.cfg</source>
#* Add these lines in the file. Check the manual for more options.
#: <source>
DataDir=/local2/public_arthor
MaxConcurrentSearches=6
MaxThreadsPerSearch=8
AutomaticIndex=false
AsyncHitCountMax=20000
Depiction=./depict/bot/svg?w=%w&h=%h&svgunits=px&smi=%s&zoom=0.8&sma=%m&smalim=1
Resolver=https://sw.docking.org/util/smi2mol?smi=%s
</source>
=== Start Arthor Instance ===
# Now ssh into a machine you wish to run an Arthor instance on and become root
# Change your shell to bash if you havn't already
#: <source>bash</source>
# Create a screen
#: <source>screen -S <screen_name></source>
# Prepare Arthor Config Path
#: <source>export ARTHOR_CONFIG="/nfs/soft2/arthor_configs/arthor-<version>/<name_of_config_file>.cfg"</source>
# Launch java
#: <source>java -jar -Dserver.port=<port_number> /nfs/soft2/arthor_configs/arthor-<version>/arthor-server-<version>.war</source>

=== Configuration Details ===

*'''DataDir''': This is the directory where the Arthor data files live. Location where the index files will be created and loaded from.

*'''MaxConcurrentSearches''': Controls the maximum number of searches that can be run concurrently by setting the database pool size. When switching between a large number of databases it can be useful to have a larger pool size, the only trade off is keeping file pointers open.

*'''MaxThreadsPerSearch''': The number of threads to use for both ATDB and ATFP searches

*Set '''AutomaticIndex''' to false if you don't want new smiles files added to the data directory to be indexed automatically

*'''AsyncHitCountMax''': The upper-bound for the number of hits to retrieve in background searches.

*'''Resolver''': Using Smallworld API, allows input box to take in a SMILE format and automatically draw on the board.

'''Check Arthor manual for more configuration options'''

== How to Build Arthor Databases==
We can build Arthor Databases anywhere. Consolidate smiles into one directory so you can index them all one by one.

Just use the script located at '''/nfs/home/jjg/scripts/arthor_index_script.sh''' at the directory where you stored the smiles.

Here is the content of the script:
<source>
#!/bin/bash

target="*.smi"
threads='48'
version="3.6.1"

export ARTHOR_DIR=/nfs/soft2/arthor_configs/arthor-$version/arthor-$version-centos8/
export PATH=$ARTHOR_DIR/bin/:$PATH

for j in $target
do
echo 'smi2atdb -v -r -j '$threads' -p '$j' '${j}'.atdb |& tee -a '${j}'_log'
time smi2atdb -v -r -j $threads -p $j ${j}.atdb |& tee -a ${j}_log
echo 'smi2atfp -j '$threads' -p '$j' '${j}.atfp' |& tee -a '${j}'_log'
time smi2atfp -j $threads -p $j ${j}.atfp |& tee -a ${j}_log
done

for i in $target*atdb
do
echo 'atdb2fp -j '$threads' '$i
time atdb2fp -j $threads $i
chmod 744 $i.fp
done
</source>

=== Command Details ===
'''smi2atdb''' creates the atdb files needed for Substructure searching.
*'''-v''' verbose
*'''-r''' Compute and store ring size information in the index allowing ring size (3-10) expressions ([r6] and [Z5])
*'''-j''' is the amount of threads to use to index the smiles file
*'''-p''' stores the position of the original file
'''atdb2fp''' makes substructure searching faster

'''smi2atfp''' creates atfp files for similarity searching

==Setting up Round Table==
"Round Table allows you to serve and split chemical searches across multiple host machines. The implementation provides a lightweight proxy that forwards requests to other Arthor host servers that do the actual search. Communication is done using the existing Web APIs.
===Setting up Host Server===
# Ssh to nfs-soft2 and become root
# Open a port in the firewall
#: <source>firewall-cmd --permanent --add-port=<port_number>/tcp
firewall-cmd --reload
</source>
# Go to Arthor Config Directory
#: <source>cd /export/soft2/arthor_configs/arthor-<version></source>
# Create Round Table Head configuration file. Here is an example:
# <source>
[RoundTable]
RemoteClient=http://10.20.0.41:8008
RemoteClient=http://10.20.5.19:8008
Resolver=https://sw.docking.org/util/smi2mol?smi=%s
</source>
# Now ssh into a machine you wish to run the round table head on and become root
# Change your shell to bash if you havn't already
#: <source>bash</source>
# Create a screen
#: <source>screen -S <screen_name></source>
# Prepare Arthor Config Path
#: <source>export ARTHOR_CONFIG="/nfs/soft2/arthor_configs/arthor-<version>/<round_table_head>.cfg"</source>
# Launch java
#: <source>java -jar /nfs/soft2/arthor_configs/arthor-<version>/arthor-<version>-centos7/java/arthor.jar --httpPort=<port_number></source>

== Active Arthor Instances ==
===Public Arthor===
{| class="wikitable"
|-
! Rocky Linux Machine
! Port
! Data Directory
! Which Arthor
|-
| arthor
| 10.20.200.100:8080
| /local3/public_arthor/
| Public Arthor
|-
|}

===Private Arthor===
{| class="wikitable"
|-
! Rocky Linux Machine
! Port
! Data Directory
! Which Arthor
|-
| arthor
| 10.20.200.100:8081
| /local3/private_arthor/
| Private Arthor
|-
|}

===Super Private Arthor===
{| class="wikitable"
|-
! Rocky Linux Machine
! Port
! Data Directory
! Which Arthor
|-
| arthor
| 10.20.200.100:8082
| /local3/super_private_arthor/
| Private Arthor
|-
|}

===Arthor BB, CC, and Covalent===
{| class="wikitable"
|-
! CentOS 7 Machine
! Port
! Data Directory
! Which Arthor
|-
| epyc-A40
| 10.20.200.92:8081
| /local2/arthorbb
| ArthorBB
|-
| epyc-A40
| 10.20.200.92:8082
| /local2/arthorcc
| ArthorCC
|-
| epyc-A40
| 10.20.200.92:8080
| /local2/arthorcovalent
| Arthor Covalent
|-
|}

== Customizing Arthor Frontend To Our Needs (Arthor 4.1) ==
'''These instructions only worked and compiled for me in the machine called epyc which is running Rocky 8 Linux operating system.'''

Summary of changes in index.html:
*Add contact info
*Advertise TLDR
*Remove button for Formula
*Database Information Link

Summary of changes in index.js:
*Hyperlink the results to zinc20 or cartblanche22
*Copy SMILES with clipboard button

Summary of changes in sketcher.js:
*Input box should be updated as user draws molecule

Summary of changes in arthor-swagger.yaml.js:
*Make API point to "/"

=== Install Prerequisite Packages ===
# Install Apache Maven
#* <source>dnf install maven -y</source>
# Install Node Package Manager (NPM)
#* <source>dnf install npm -y</source>
# In your home directory, create a new directory to hold the files for the upcoming procedures
#* <source>mkdir /mnt/nfs/home/jjg/arthor_build_from_source</source>
# Download these latest Arthors and store them in 'arthor_build_from_source/'. [[Arthor Documentation for Future Developer#How To Download Arthor|Here's how to download Arthor.]]
#* arthor-4.1-source.tar.gz
#* arthor-4.1-centos8.tar.gz
# Extract contents from the tar.gz files
#* <source>tar -xvf arthor-4.1-source.tar.gz
tar -xvf arthor-4.1-centos8.tar.gz</source>
# Install Apache Maven Arthor dependencies through this script located at '''nfs/home/jjg/arthor_build_from_source/mvn_build.sh'''
#* <source>
#!/bin/bash

version=4.1

export ARTHOR_DIR=/mnt/nfs/home/jjg/arthor_build_from_source/arthor-$version/arthor-$version-centos8/java/
export OS=linux
export VERSION=$version

mvn install:install-file -Dfile=$ARTHOR_DIR/arthor-jni-${OS}.jar \
-Dpackaging=jar \
-DgeneratePom=true \
-DartifactId=arthor-jni-${OS} \
-DgroupId=com.nextmovesoftware.arthor \
-Dversion=$VERSION
mvn install:install-file -Dfile=$ARTHOR_DIR/arthor-jni.jar \
-Dpackaging=jar \
-DgeneratePom=true \
-DartifactId=arthor-jni \
-DgroupId=com.nextmovesoftware.arthor \
-Dversion=$VERSION
</source>

=== Customizing Index.html ===
Location: /nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server-ui/src/index.html

==== Add contact info and tldr ====
# vim /nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server-ui/src/index.html
# Search for 'arthor_table_list'
#: <source>?arthor_table_list</source>
# Add contact info, a plug to tldr.docking.org, and database link after the div block where arthor_table_list is inside of. It should look like this
#: <source lang="html">
<div class="opt-box-border">
<label>Databases</label>

<ul id="arthor_table_list">
<li class="placeholder">Please select a search type</li>
</ul>
</div>
<div class="opt-box-border">
<label>Ask Questions</label>
Email us: jjiteam@googlegroups.com
</div>
<div class="opt-box-border">
<label> To Download 100K+ Results</label>
Sign up for <a href="http://tldr.docking.org/">tldr.docking.org</a> and use arthorbatch
</div>
<div class="opt-box-border">
<label> Database Information</label>
<a href="https://wiki.docking.org/index.php?title=Arthor_Databases" target="_blank">Database Information Link</a>
</div>
</source>

==== Remove Similarity and Formula Buttons ====
# vim /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server-ui/src/index.html
# Search for 'arthor_search_list'
#: <source> ?arthor_search_list </source>
# Delete and Replace the whole 'ul' element block with this
#: <source lang="html">
<li value="Similarity" onclick="setSearchType(this)" class="first">
Similarity
</li><li value="Substructure" onclick="setSearchType(this)">
Substructure
</li><li value="SMARTS" onclick="setSearchType(this)" class="last">
SMARTS
</li>

</source>

=== Customize Index.js ===
==== Conditional Hyperlinks ====
Location /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1-source/server-ui/src/index.js

# Add the zinc20 hyperlink to the Arthor results. Search for this
#: <source lang="javascript">
$info.append($("<tr>").append($('<td class="identifier">').append(id)));
</source>
# Delete the contents of the whole else block and replace it with this
#: <source lang="javascript">
if (id.substring(0, 6) == "ZINC00") {
$info.append(
$('<tr>').append(
$('<td class="identifier">').append("<b><a target='_blank' href='https://zinc20.docking.org/substances/"+id+"'>" + id + "</a></b>")));
$info.append(
$("<tr>").append(
$("<td>").append(
"<span class='table_name'>" + table_name + "</span>",
),
),
);
} else if (id.substring(0,4) == "ZINC" && id.substring(4,6) != "00") {
$info.append(
$('<tr>').append(
$('<td class="identifier">').append("<b><a target='_blank' href='https://cartblanche22.docking.org/substance/"+id+"'>" + id + "</a></b>")));
$info.append(
$("<tr>").append(
$("<td>").append(
"<span class='table_name'>" + table_name + "</span>",
),
),
);
} else {
$info.append($("<tr>").append($('<td class="identifier">').append(id)));
$info.append(
$("<tr>").append(
$("<td>").append(
"<span class='table_name'>" + table_name + "</span>",
),
),
);
}
</source>
==== Copy SMILES Clipboard ====
# Search this
#: <source>?parser</source>
# At the end of the the try block add this
#: <source lang="javascript">
$info.append(
$('<tr>').append(
$('<td>').append('Copy SMILES: <a href="#" onclick="navigator.clipboard.writeText(\'' + parser.smi + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>')));
</source>
# The whole try and catch block should look like this
#: <source lang="javascript">
try {
var parser = new SmiJs.Parser(data);
if (parser.parse()) {
$info.append(
$('<tr>').append(
$('<td>').append(parser.mf(true))));
$info.append(
$('<tr>').append(
$('<td>').append(parser.mw())));
$info.append(
$('<tr>').append(
$('<td>').append('Copy SMILES: <a href="#" onclick="navigator.clipboard.writeText(\'' + parser.smi + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>')));
}
} catch(err) {
console.log(err);
}
</source>

=== Customize Sketcher.js ===
Location: /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1-source/server-ui/src/sketcher.js

==== Input Box Updates as User Draws ====
#vim /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1-source/server-ui/src/sketcher.js
#Search for this line "var smiles = event.src.smiles();"
#* <source>?var smiles = event.src.smiles();</source>
#Add this new line below it
#* <source lang="javascript">$('#ar_text_input').val(smiles);</source>

=== Customize arthor-swagger.yaml ===
Location: /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server/src/main/webapp/WEB-INF/static/swagger/arthor-swagger.yaml

==== Change URL Base Path for API Call ====
# vim /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server/src/main/webapp/WEB-INF/static/swagger/arthor-swagger.yaml
# Search for "servers"
#* <source>?basePath</source>
# Remove "arthor" and keep argument as "/"

=== Compile/Minify Code through NPM ===
# Install NPM packages and Minify Code
#* <source>
cd /nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server-ui
npm install
npx webpack-dev-server
npx webpack --mode=production
</source>
# If the npx commands don't work, try updating node and npm version to the latest one.
# Build the war file
#* <source>
cd /nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server
mvn install -Pbootable
</source>
# If it works, then the path to new war file is at /mnt/nfs/home/jjg/arthor_build_from_source/arthor-4.1/arthor-4.1-source/server/target/arthor-server-4.1.war

== Adding Arthor to Systemd (example) ==
# Create service files
#: <source>
touch /etc/systemd/system/arthor.service
</source>
# Add this code block in the service file
#: <source>

[Unit]
Description=Arthor Public Instance
After=network.target

[Service]
Type=exec
Restart=always
RestartSec=3
User=root
ExecStart=/usr/bin/bash /nfs/soft2/arthor_configs/start_arthor_service.sh -arthor
ExecStop=/bin/kill -15 $MAINPID

[Install]
WantedBy=multi-user.target
</source>
# Change "ExecStart" to other Arthor Instances

== Restarting Arthor Instance(s) Instructions ==
=== Public, Private Arthor, and Super Private Arthor ===
# ssh to "arthor" server and run
#: <source>
systemctl <start|stop|restart> <arthor|arthorp|arthorc>
</source>

=== ArthorBB, ArthorCC, and ArthorCovalent ===
# ssh to "epyc-a40" server and run
#: <source>
systemctl <start|stop|restart> <arthorbb|arthorcc|arthorcovalent>
</source>

[[Category:Arthor]]
[[Category:Developer]]

Cluster Networking Quirks

2025-05-05T22:58:58Z

Jgutierrez6: Created page with "== Introduction == This is a wiki page detailing weird network quirks the lab has encountered over the years. == List of events == *May 5, 2025 *: <source> Vav and Gimel (both are in the 169.230.26.0 subnet) had a problem contacting the 169.230.75.0 subnet. Both of these machines had similar ip route rules, displayed below. #vav 169.230.0.0/16 dev enp4s0 proto kernel scope link src 169.230.26.169 metric 100 #gimel 169.230.0.0/16 dev eth4 proto kernel scope link src..."

== Introduction ==
This is a wiki page detailing weird network quirks the lab has encountered over the years.

== List of events ==
*May 5, 2025
*: <source>
Vav and Gimel (both are in the 169.230.26.0 subnet) had a problem contacting the 169.230.75.0 subnet.
Both of these machines had similar ip route rules, displayed below.

#vav
169.230.0.0/16 dev enp4s0 proto kernel scope link src 169.230.26.169 metric 100
#gimel
169.230.0.0/16 dev eth4 proto kernel scope link src 169.230.26.166

The rules above tell us that any IP addresses in this range 169.230.1.0 ~ 169.230.255.254 should be visible but they weren't.

But for some reason, if I narrow down the subnet to 255.255.255.0 then we're able to ping the 169.230.75.0 subnet. So here are the new rules that allow connection to happen.

#vav
169.230.26.0/24 dev enp4s0 proto kernel scope link src 169.230.26.169 metric 100
#gimel
169.230.26.0/24 dev eth4 proto kernel scope link src 169.230.26.166
</source>

Globus Curator's Guide

2025-03-19T22:37:29Z

Jgutierrez6:

== Introduction ==
Here will be the instructions on how to maintain globus.

=== Notes ===
*For any restrict path's to work in the Guest Collections, the main collection needs to have either the same or broader path to be allowed.
*The main collection is currently private. Only the admins can have access to it.

== Updating Path Permissions for the Guest Collection ==
#Identify the real paths of the permission denied paths. Globus doesn't follow symbolic links
#:<source>readlink -f <symlink> </source>
# Become root in epyc
##Go to /root/globus
#:<source>cd /root/globus</source>
#Insert and save the real paths at the end of this file (check commas for syntax or you'll get an error):
#:<source>vim storage_gateway_path_restriction.json</source>
##Here is an example of what it should look like
##:<source>
{
"DATA_TYPE": "path_restrictions#1.0.0",
"none": ["/"],
"read": [
"/nfs/ex3/published",
"/nfs/exe/work/jji/2dexport",
"/nfs/exe/work/jji/3d-zinc20",
"/nfs/exl/zinc-22a",
"/nfs/exl/zinc-22b",
"/nfs/exl/zinc-22c",
"/nfs/exc/zinc-22n"
],
"read_write":[
"/nfs/mammoth/idg-images/newAmisData"
]
}
</source>
#Create a session token with globus. (Cluster 2 Password, ucsfbks)
#:<source>globus-connect-server session update ucsfbks@globusid.org</source>
#Follow instructions given by globus and it should tell you that it was successful
#In the same directory, run this command to update
#:<source>bash globus-storage-gateway.sh update</source>
#This is the successful message
#:<source>Message: Updated Storage Gateway aa731fb1-b27a-49e6-aaeb-2d2cc252f779</source>

== Updating Path Permissions for the Main Collection ==
#Identify the real paths of the permission denied paths. Globus doesn't follow symbolic links
#:<source>readlink -f <symlink> </source>
# Become root in epyc
##Go to /root/globus
#:<source>cd /root/globus</source>
#Insert and save the real paths at the end of this file (check commas for syntax or you'll get an error):
#:<source>vim path_restrictions.json</source>
##Here is an example of what it should look like
##:<source>
{
"DATA_TYPE": "path_restrictions#1.0.0",
"read": [
"/"
],
"read_write": [
"/nfs/mammoth/idg-images/newAmisData/"
]
}
</source>
#Create a session token with globus. (Cluster 2 Password, ucsfbks)
#:<source>globus-connect-server session update ucsfbks@globusid.org</source>
#* If above doesn't work, run below and follow instructions. (Cluster 2 Password, ucsfbks)
#:: <source> globus-connect-server login localhost </source>
#Follow instructions given by globus and it should tell you that it was successful
#In the same directory, run this command to update
#:<source>globus-connect-server collection update --sharing-restrict-paths file:path_restrictions.json 2e1e1781-6d8a-440a-8604-7100d9c65c90</source>
#This is the successful message
#:<source>code: success</source>
#Lastly, in the globus web GUI, you need to allow a specific user or group to access the new path in the permissions tab located in the collection overview.