Profiling Data: understanding Markov dataShape

#1

I can see the potential in “profileData()” method. However, I’m trying to fully understand what the markov datashape output is telling me. I get that it’s listing out the various possible combinations found. While I don’t truly think it’s actually markovian, that’s a different topic. My main question revolves around the meanings behind the notation. For example, what indicated by the numbers following each of the strings within the lists (e.g.: what does the 17 mean in [‘R’ 17,…]?)?

markov: order 2 length 3
["" [‘R’ 17, ‘L’ 13, ‘N’ 11], “RS” [‘7’ 17], “R” [‘S’ 17], “L1” [‘S’ 7], “NT” [‘2’ 1, ‘7’ 6, ‘8’ 2], “N4” [‘8’ 2], “L4” [‘8’ 1], “L7” [‘9’ 3], “L” [‘1’ 7, ‘4’ 1, ‘7’ 3, ‘9’ 2], “L9” [‘1’ 2], “N” [‘T’ 9, ‘4’ 2]]

In this particular dataset, the number of non-Null records for this field is 92193, while cardinality is 625814. This particular field lists “null: 98.0%”. That doesn’t look right, either.

Thanks for any help you can provide.

0 Likes