Figure-Eight-Knots

1. Inquiry #2 - What Fieldmarks Associate with Figure 8 Knots

Which KFG Fieldmarks lie near figure 8 knots, indicating a causal relationship between Figure-8-Knots and other types of KFG Fieldmarks such as subsidiary count, or pendant-pendant-sum count. To calculate this, we use the Khipu Fieldmark Table used to generate an overall view of the KFG Fieldmarks, and turn into a column of unit vectors suitable for processing by the cosine-distance sorter

Code

```{python}
import os
import numpy as np
from scipy import stats
import pandas as pd
from pandas import Series, DataFrame
from collections import Counter

import qollqa_chuspa as qc  # A Khipu Maker is known (in Quechua) as a Khipu Kamayuq
import utils_loom as uloom
import utils_khipu as ukhipu
import utils_pandas as upanda
import utils_kfg_locations as uloc
    
# Plotly
import plotly
from plotly.offline import iplot, init_notebook_mode
import plotly.graph_objects as go
import plotly.express as px
import plotly.figure_factory as ff
plotly.offline.init_notebook_mode(connected = False)
# Load all khipus
(khipu_dict, all_khipus) = qc.fetch_khipus()

KFG_Names = ukhipu.kfg_order(khipu_dict.keys())
khipu_cords_dict = {KFG_Name: khipu_dict[KFG_Name].all_cords(include_bottom_cords=True, include_top_cords=True, include_subsidiaries=True) for KFG_Name in KFG_Names}
total_num_khipu_cords = sum([len(khipu_cords_dict[KFG_Name]) for KFG_Name in KFG_Names])
all_khipu_cords = uloom.flatten_list([khipu_cords_dict[KFG_Name] for KFG_Name in KFG_Names])
```

Code

```{python}
import os
import numpy as np
from scipy import stats
import pandas as pd
from pandas import Series, DataFrame
from collections import Counter

import qollqa_chuspa as qc  # A Khipu Maker is known (in Quechua) as a Khipu Kamayuq
import utils_loom as uloom
import utils_khipu as ukhipu
import utils_pandas as upanda
import utils_kfg_locations as uloc
    
# Plotly
import plotly
from plotly.offline import iplot, init_notebook_mode
import plotly.graph_objects as go
import plotly.express as px
import plotly.figure_factory as ff
plotly.offline.init_notebook_mode(connected = False)
# Load all khipus
(khipu_dict, all_khipus) = qc.fetch_khipus()
KFG_Names = ukhipu.kfg_order(khipu_dict.keys())

khipu_cords_dict = {KFG_Name: khipu_dict[KFG_Name].all_cords(include_bottom_cords=True, include_top_cords=True, include_subsidiaries=True) for KFG_Name in KFG_Names}
all_khipu_cords = uloom.flatten_list([khipu_cords_dict[KFG_Name] for KFG_Name in KFG_Names])
total_num_khipu_cords = len(all_khipu_cords)
```

Code

```{python}
eight_knot_cords_df = pd.read_csv(f"{uloc.fieldmarks_data_dir()}/eight_knot_cords.csv")

sole_eight_knots_df= eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Sole_Eight_Knot']
trailing_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Trailing_Eight_Knot']
only_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Only_Eight_Knot']
leading_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Leading_Eight_Knot']
middle_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Middle_Eight_Knot']
mixed_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Mixed_Eight_Knot']

the_num_eight_knot_khipus = eight_knot_cords_df['KFG_Name'].nunique()
the_num_sole_eight_knot_khipus = sole_eight_knots_df['KFG_Name'].nunique()
the_num_trailing_eight_knot_khipus = trailing_eight_knots_df['KFG_Name'].nunique()
the_num_leading_eight_knot_khipus = leading_eight_knots_df['KFG_Name'].nunique()
the_num_middle_eight_knot_khipus = middle_eight_knots_df['KFG_Name'].nunique()
the_num_mixed_eight_knot_khipus = mixed_eight_knots_df['KFG_Name'].nunique()

the_num_eight_knot_cords = eight_knot_cords_df.shape[0]
the_num_sole_eight_knot_cords = sole_eight_knots_df.shape[0]
the_num_trailing_eight_knot_cords = trailing_eight_knots_df.shape[0]
the_num_multiple_only_eight_knot_cords = only_eight_knots_df.shape[0]
the_num_leading_eight_knot_cords = leading_eight_knots_df.shape[0]
the_num_middle_eight_knot_cords = middle_eight_knots_df.shape[0]
the_num_mixed_eight_knot_cords = mixed_eight_knots_df.shape[0]

sole_eight_knots_df= eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Sole_Eight_Knot']
trailing_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Trailing_Eight_Knot']
leading_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Leading_Eight_Knot']
middle_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Middle_Eight_Knot']
multiple_only_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Multiple_Only_Eight_Knots']
mixed_eight_knots_df = eight_knot_cords_df[eight_knot_cords_df['tag'] == 'Mixed_Eight_Knot']

def num_kfg_cords(kfg_name, df): return len(df[df['KFG_Name'] == kfg_name])
num_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, eight_knot_cords_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, eight_knot_cords_df) > 0})
num_sole_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, sole_eight_knots_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, sole_eight_knots_df) > 0})
num_trailing_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, trailing_eight_knots_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, trailing_eight_knots_df) > 0})
num_leading_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, leading_eight_knots_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, leading_eight_knots_df) > 0})
num_mixed_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, mixed_eight_knots_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, mixed_eight_knots_df) > 0})
num_middle_eight_knots_dict = uloom.sort_dict_by_values({KFG_Name: num_kfg_cords(KFG_Name, middle_eight_knots_df) for KFG_Name in KFG_Names if num_kfg_cords(KFG_Name, middle_eight_knots_df) > 0})
```

Code

```{python}
# Read in the Fieldmark and its associated dataframe and match dictionary
from fieldmark_figure8knots import FieldmarkFigure8Knots
figure8knot_fieldmark = FieldmarkFigure8Knots()
figure8_knot_fieldmark_df = figure8knot_fieldmark.fieldmark_df().sort_values('num_8knot_cords', ascending=False)
figure8_knot_fieldmark_df.sort_values('num_8knot_cords', ascending=False, inplace=True)   
if 'Unnamed: 0' in figure8_knot_fieldmark_df.columns: figure8_knot_fieldmark_df.drop(columns=['Unnamed: 0'], inplace=True)                                                  
raw_match_dict = figure8knot_fieldmark.raw_match_dict()

```

Code

```{python}
## Read in the entire database of fieldmarks, 
## Then clean up columns and add additional columns for figure-8 knots
## Then unit-normalize the fieldmarks to get it ready for cosine similarity
from fieldmark_table import KhipuFieldmarkTable
def build_augmented_fieldmarks_df():
    # Fetch the fieldmarks dataframe
    the_fieldmark_df = KhipuFieldmarkTable().fieldmark_df()
    assert the_fieldmark_df is not None
    the_KFG_Names = the_fieldmark_df['kfg_name'].tolist()
    the_fieldmark_df.sort_values(by='mean_cord_value', ascending=True, inplace=True)
    the_fieldmark_df.set_index('kfg_name', inplace=True)
    the_fieldmark_df.drop(columns=['similarity_index', 'cord_group_count', 'fanout_ratio', 'figure8knots',
                                   #'pendant_cord_count', s_cords',
                                   'equal_sums', 'multi_summands'], inplace=True)
    
    # Append Values for Eight-Knots
    the_fieldmark_df['num_eight_knot_cords'] = uloom.to_unit_vector([num_eight_knots_dict.get(kfg_name, 0) for kfg_name in the_KFG_Names])
    the_fieldmark_df['num_sole_eight_knot_cords'] = uloom.to_unit_vector([sole_eight_knots_df[sole_eight_knots_df['KFG_Name'] == kfg_name].shape[0] for kfg_name in the_KFG_Names])
    the_fieldmark_df['num_trailing_eight_knot_cords'] = uloom.to_unit_vector([trailing_eight_knots_df[trailing_eight_knots_df['KFG_Name'] == kfg_name].shape[0] for kfg_name in the_KFG_Names])
    the_fieldmark_df['num_leading_eight_knot_cords'] = uloom.to_unit_vector([leading_eight_knots_df[leading_eight_knots_df['KFG_Name'] == kfg_name].shape[0] for kfg_name in the_KFG_Names])
    the_fieldmark_df['num_middle_eight_knot_cords'] = uloom.to_unit_vector([middle_eight_knots_df[middle_eight_knots_df['KFG_Name'] == kfg_name].shape[0] for kfg_name in the_KFG_Names])

    include_equal_sums = True
    if include_equal_sums:
        equal_sums_fieldmark_df = pd.read_csv(f"{uloc.fieldmarks_data_dir()}/equal_sums.csv")
        assert equal_sums_fieldmark_df is not None
        def num_equal_sums(kfg_name):
            return equal_sums_fieldmark_df[equal_sums_fieldmark_df['kfg_name'] == kfg_name].num_equal_sums.tolist()[0]
        equal_sums_dict = {KFG_Name: num_equal_sums(KFG_Name) for KFG_Name in KFG_Names}
        the_fieldmark_df['num_equal_sums'] = uloom.to_unit_vector([equal_sums_dict[kfg_name] for kfg_name in the_KFG_Names])
        
    return (the_KFG_Names, the_fieldmark_df) 

(fieldmark_KFG_Names, augmented_fieldmark_df) = build_augmented_fieldmarks_df()
    
#print(f"Fieldmarks = {nl_char}{uloom.multiline(augmented_fieldmark_df.columns.tolist(), split_char=',', continuation_char=','+nl_char)}")
#print(f"Unit-Normalized Augmented Fieldmarks for Cosine Similarity")
#augmented_fieldmark_df.head(5)
```

Code

```{python}
# Create Cosine Similarity Matrix
import utils_pandas as upanda
nl_char = "\n\t"

flowmap_df = figure8_knot_fieldmark_df.copy(deep=True)
cosine_sim_matrix = upanda.CosineSimilarityMatrix(augmented_fieldmark_df, start_column="num_eight_knot_cords", start_row="KH0239")
sorted_df = cosine_sim_matrix.sort(sort_columns = True, sort_rows = True)
sorted_columns = sorted_df.columns.tolist()
print(f"Fieldmarks sorted by Closest Match using Cosine Similarity = {nl_char}{uloom.multiline(sorted_columns, split_char=',', continuation_char=nl_char)}")
      
fig = (px.imshow(sorted_df.to_numpy().tolist(),
                labels=dict(x="Fieldmark", y="Khipu Name", ),
                x=sorted_df.columns,
                y=fieldmark_KFG_Names,
                width=944, height=2000, aspect="auto")
        .update_coloraxes(showscale=False)
        .update_layout(
                title= 'Fieldmark Features - Sorted by Cosine Similarity - Hover over line for more info',
                xaxis= dict(tickangle=270),
                xaxis_nticks=5000)
        .show())
```

Fieldmarks sorted by Closest Match using Cosine Similarity = 
    ['num_eight_knot_cords', 'num_sole_eight_knot_cords', 'benford_match'
     'max_cord_level', 'ascher_color_count', 'num_cords', 'pendant_cord_count'
     'pendant_pendant_sum', 'colored_pendant_sum', 'indexed_pendant_sum'
     'color_bands', 's_cords', 'subsidiary_cord_count', 'mean_cords_per_group'
     'percent_z_knots', 'recto_ratio', 'percent_s_knots', 'verso_ratio'
     'num_trailing_eight_knot_cords', 'num_equal_sums'
     'num_leading_eight_knot_cords', 'num_middle_eight_knot_cords'
     'mean_cord_value', 'indexed_subsidiary_sum', 'subsidiary_pendant_sum'
     'pendant_sub_neighbor', 'group_sum_bands', 'ascher_decreasing_group', 'z_cords'
     'group_group_sum', 'double_sum_top_cords', 'sum_top_cords', 'top_cords']

The column sorting reveals the association of figure-eight-knot cords with other khipu fieldmarks.

Set 1:

Num Eight Knot Cords
Num Sole Eight Knot Cords

Set 2:

Benford Match
Max Cord Level
Ascher Color Count
Num Cords
Pendant Cord Count

Set 3:

Pendant Pendant Sum
Colored Pendant Sum
Indexed Pendant Sum
Group Group Sum

Set 4:

Color Bands
S Cords
Subsidiary Cord Count
Mean Cords per Group
Percent Z Knots
Recto Ratio
Percent S Knots
Verso Ratio

Set 5:

Num Trailing Eight Knot Cords
Num Equal Sums
Num Leading Eight Knot Cords
Num Middle Eight Knot Cords

Set 6:

Top Cords
Sum Top Cords
Double Sum top Cords
Ascher Decreasing Group
Subsidiary Pendant Sum
Indexed Subsidiary Sum
Pendant Sub Neighbor
Z Cords
Mean Cord Value

Observations:

Set’s 1 and 5 are the Figure-8-Knot sets.
Set 1 is interesting due to the 2nd Fieldmark - Sole-8-Knots. Sole-8-Knots were present in 68% of all cords with a Figure-8-Knot, and are distributed 52%/48% over pendants/subsidiaries.
Set 2 is the Benford Match set, indicating that Sole-8-Knots match with highly numeric khipus. Benford Match khipus are strongly associated with numeric khipus (as opposed to narrative khipus), and this tells us that Figure-8-Knots are associated with numeric, not narrative khipus.
Set 3 are Ascher sums. Here is the first evidence that Sole-8-Knots have a causal relationship with Ascher sums - especially Pendant, Colored, and Indexed Pendant Sums.
Set 5 comprises the 32% of the Figure-8-Knot Cords that are mostly Lockean sequences ie. 1 as a last digit Figure-8-Knot (with the rare exception of Leading and other Figure-8-Knot types). Interestingly, Dual/Balanced Sums show in coassociation with trailing/leading/middle Figure-8-Knots.