the last installment found a way to construct large “ufos” in the postdetermined region and in crisp clinical terms refuted some longrunning hypotheses, in more informal pov “throwing a wrench in the works” of big overall proof strategies painstakingly built up over years and showing both limitations and strengths of the overall empirical/ datamining approach. however, maybe there is a silver lining; there was a larger hypothesis that maybe escapes largely unscathed, ie in more dramatic terms can yet be rescued amidst some of the substantial wreckage/ smoking ruins (aka “easy come, easy go”™). a recent experiment found that measuring “slope correlation” in the postdetermined region gives high adherence to linearity (ie close to 1) even as iterates get larger. this was discovered with
bitwise optimization and never did convert that finding to the stronger/ more rigorous
hybrid optimization and it was in the back of my mind to do that.
looked over the hybrid code and had the urge to refactor it. the bit vector initialization was servicable but really funky. also had an idea to extend vectors at a few bit positions at a time, not just at the msbs. also came up with the idea of a corresponding/ symmetric shortening operator.
the basic experiment here was trying to minimize slope correlation ‘sc’ for larger iterates. however the naive code simply found a significantly low ‘sc’ for some small iterates and then didnt get past those smaller iterates, the search got stuck so to speak. while this finding is consistent with the hypothesis my real question was what the trend for ‘sc’ was for higher iterates. so then bit the bullet and did multiparameter optimization for this hybrid code, something that is novel wrt prior code/ experiments also. the multiparameter optimization is based on gaussian normalization recalculated every 100 samples. the optimization is to push up ‘nw’ the iterate bit width and push down ‘sc’ the slope correlation. the code reaches very high iterate sizes ~4.5K and yet cant push down ‘sc’ more than ~0.9684. the multioptimization means that there are smaller iterates with smaller slope correlations, but the algorithm moves past those as pushed to search higher iterates. so this general observation/ trend still seems “robust”.