CHANGE HISTORY FOR Djinn -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- E - Evaluation change S - Alteration to the search I - Interface change (a command was added, deleted etc.) B - Bug fix O - Optimization 14-APR-06 (Ver. 0.925x) ----------------------- S o No moves are reduced if we are within two ply of the horizon (as determined by the "ply" variable. Note, for Djinn the "ply" variable corresponds to everyone else's "depth" variable). 02-APR-06 --------- S o More tweaks to the time management algorithms. S o Refactored the KPK recognizer functions to be more readable. 26-MAR-06 --------- B o Added additional debugging code to detect time management problems. I also *completely* restructured the way the CPU time is calculated when the engine starts up, so it should be much more accurate now. 23-MAR-06 --------- I o Added an option to the configuration file to allow the user to set the size of the evaluation hash table. Thanks to Gábor Szots for pointing this out to me. 18-MAR-06 (Ver. 0.920) ---------------------- E o Revived (and optimized) the "dangerous pawns on the 6th" code. The program was missing too many situations where the two connected passed pawns were stronger than the rook or knight. This code isn't 100% correct, but it's damn close! B o Corrected a bug in the way passed pawns were scored. Occasionally, if two pawns were doubled and the front pawn was passed, the engine would also evaluate the back pawn as passed, which is incorrect. B o Caught a *major* bug in the KPK internal-node recognizer code. The value returned from the KPK recoginizer was not being negated if black had the right to move (very bad!). This was actually costing the engine a fair number of endgame points. 12-MAR-06 --------- S o Changed the extension for pawn pushes to the 6th and 7th to 3/4 of a ply instead of 1/2. This helped the program's pawn endgame a bit. O o Added the "auto-generated" attack functions back in. This resulted in an approximate speed up of about 5%. 09-MAR-06 --------- S o The engine now restricts all null move searches until we are at least 3 ply deep into the tree. This helps the program avoid Qxe3+ in this position 8/8/1Q6/4kp2/P4p1P/4r3/8/6K1 w - - 0 1 or Bxg5 in this position 4rbk1/pb5p/1p1r1p2/5PpN/1P2q3/4B3/PQ4PP/1R1R2K1 w - - 0 16 S o Additional tweaks to the late-move reduction criteria. Removed the influence of the history table, so the best name *really* is "late-move" and not "history" reductions (at least for Djinn)! ;-) Also the engine now uses a form of "adaptive" late-move reductions, which seems to give it an extra half-ply of search depth, with no real downside (at least so far). S o Threw out the "recapture" extension. This helps in test suites, but in my testing doesn't add anything to actual game results. E o Tweaks to the pawn piecetables (nothing major, just a few changes to encourage the program to favor duos over lone adventuring pawns). 05-MAR-06 --------- S o Additional late-move reductions. The program pays far more attention now to the Knuth node type, essentially using this information to decide on wheter a late move reduction is warranted, (this modification was added after a discussion with Daniel Shawul on the Winboard forum). Also a modification to the way extensions are scaled back, if the side moving has a huge material advantage the extensions are less aggresive. B o Fixed a couple of minor bugs in the way test results were reported, having to do with integer versus floating point conversion. 11-FEB-06 --------- S o Modified (i.e. simplified) the times that the engine will generate checks in the qsearch. Eliminated the "always_in_check" variable since it was redundant after the refactoring. 04-FEB-06 --------- S o Modified the futility pruning in the qsearch so that the margin was slightly larger for pruning captures (I modified this based on scaffolding code that measured the effectiveness of various margins). 21-JAN-06 --------- S o Restructed the search in both the main tree and the qsearch to be a "pure" PVS search. E o A plethora of king-safety tweaks (and some simplification of the code since it was becoming overally messy). 29-DEC-05 --------- S o Restored the internal-iterative deepening, but it is heavily restricted based on the node type. It is only performed at PV nodes when we don't have a hash move. 26-DEC-05 --------- B o Caught a rather subtle bug in the way the draft was being calculated in the quiescent search. Previously, "-qdepth*ONE_PLY" was being used as the draft for the qsearch. Now "ply" is passed to the qsearch as well as the regular search and decremented each time by "ONE_PLY". This fixed an inconsistency at the horizon between the null-search and the quiescence search. E o Modified the middlegame piece table for the knights so that the absolute mobility for the knight is folded into each square. 21-DEC-05 --------- S o Added additional CHANT enhancements to help optimize the search. 13-DEC-05 --------- B o Found a bug in the king safety "Shelter" array. E o Isolani are now scored using an array with 64 values instead of the 8 entry scaling array previously used. 12-DEC-05 --------- S o Now killers from the two plies up in the tree are used if they apply to the current position. S o If the program detects that it is in check it now generates only valid check replies first (this used to come after the hash phase and killer mate phases). The hash move (if applicable) is still sorted to the top. 07-DEC-05 --------- S o Changed the way the history tables are scaled back. After adding search scaffolding code to measure the different schemes, I decide simpler was better (i.e. faster) so the scheme was simplified. 06-DEC-05 --------- S o Modified the insertion sort routine so that it is now a "stable" sort. This means that the order of duplicate values are preserved, which is important when the list is being sorted in multiple passes. S o Modified the way the history heuristic sort routine operates (say *that* five times fast)! 21-SEP-05 --------- S o Added Knuth-type node classification. I decided to revisit the whole search issue and give Djinn a search based on sound theoretical foundations--this is the main pillar of that foundation. S o Added a form of the Botvinnik-Markoff extension--slightly different then what Sergi Markoff described. S o Removed the internal-iterative deepening. This will go back in eventually, but *only* when I deem it truly effective. 12-SEP-05 --------- B o Fixed an out-of-bounds array access in the main search that would only show up in very disparate material situations. This was caught when I ran the engine in full debug mode. 11-SEP-05 --------- I o To make them easier to remember and understand, I changed the "flip/flop/flip-flop" commands to... flip-tb : Mirror the board top-to-bottom (A1=A8, A2=A7, etc.) flip-lr : Mirror the board left-to-right (A1=H8, B1=G1, etc.) flip-tblr : Perform, in succession, both of the above commands You know it's time to make an interface change when *you* have to start looking the syntax of your own commands up in the documentation. B o Caught a bug in the evalution hash code (again, I ask--how does this thing *EVER* play a legal move?? ;-) E o Modified the "eval-test" routine so that it checks the castling code. This is problematic because the castling rights get modified when the board is flip and mirrored. The solution is to artifically set the castling bonus and check the results in a consistent way. B o Caught a minor bug in the way the king positions were being score in a sparse piece endgame. B o Corrected a bug in the "rule-of-the-square" code. 02-SEP-05 --------- O o Altered the Win32 versions of "findlsb" and "findmsb", so that they are more in line with the Linux versions used with gcc. This also had the ancillary effect of speeding the Windows version up by roughly 8% (which is good of course)! 30-AUG-05 (ver. 0.884b) ----------------------- E o Tweaks to the bishop mobility code. 29-AUG-05 --------- B o Discovered a *major* castling bug. Basically, the castling bonus for black was reversed--which means the program was giving a bonus for NOT castling!! Sometimes I wonder how this thing *ever* made a legal move :-( 26-APR-04 --------- S o Ripped out the "null-move verification" search. It may have worked for Omid and Falcon but it doesn't work for Djinn. Instead the null-move is turned off completely for king and pawn only endgames or if either side has a lone king. S o Modified the value of R for null-move searches. If the previous node was reduced then we search the null-move subtree with R=2, (regardless of the value of R that the "adaptive null-move" algorithm would have set). E o Added a new KPK interior-node recognizer based on code Dieter Buerssner (of YACE fame) posted on CCC. The code is of course used with his generous permission. E o Added a new KNKP interior-node recognizer for endgames where a single knight is trying to salvage a draw against the pawn(s). S o Modified the Qsearch so that it is more restrictive when generating checking moves. E o Rewrote (i.e. simplified) the knight outpost code. 02-JUL-04 --------- E o Rolled in a reduced form of the 0.871x experimental pinned and threatened pieces code. This will more than likely need to be tuned before it will be usable. 01-JUL-04 --------- I o Added a variable to track the number of interior recognizer hits. S o Checks against the enemy king are not extended as deeply if the side moving is substantially ahead in material. 22-MAY-04 --------- B o Fixed a pondering bug that Leo discovered in his pre-CCT8 test tournaments. B o Fixed a minor bug in the way data from the search was returned to the calling FSM. One of the fields in the "info_t" return structure was incorrect. I o After numerous requests, I finally altered the way Djinn displays the time when searching. Previously, the program would show the difference in time for each *individual* ply searched. Now, it displays the cumulative time required to search the current ply, (this is in line with about 99.99% of the other engines in the world, and is thus less confusing to people). 07-MAY-04 (ver. 0.870) ---------------------- B o Minor modification to the time management code for incremental searches (due to a time forfeit against Zappa on ICS). Now, if 80% of the total search time left is less than the time allocated then the allocated time is scaled back. I also modified the algorithm so that it is more conservative if the increment is small. B o Fixed some relatively minor bugs in the way the PGN data was being saved to the log file. In addition, the PGN tags "WhiteElo" and "BlackElo" are now written to the PGN game record if known. 28-APR-04 --------- O o Optimized the static-exchange evaluation functions. They are now more accurate *and* faster (hopefully, a win-win situation)! 26-APR-04 --------- O o Modified the routine "fixup_pv()" (which extracts PV nodes out of the TT) so that it does *not* actually make the moves any longer, but instead simply updates the hash key and then uses the new key to extract the moves from the table. This is faster without any loss of information. 22-APR-04 --------- B o *Many* (many) additional assertions were added. I also created a new (and hopefully better) "board_is_ok()" function for verification of the board when in debug mode (the inspiration was Fabien Letouzey's debug code). O o Minor optimizations to the rotated bitboard code, SEE routines and bitboard "setbit/clrbit" routines. 10-APR-04 --------- S o Modified the search so that it uses ETC to order some of the moves. 08-APR-04 --------- S o Removed the (somewhat artificial) depth-based restriction on extensions. With the new move ordering it was less effective and hurt more than it helped. S o Additional restrictions on checks generated in the qsearch. S o The depth "losing mundanes" (these are non-capture that leave a piece en'prise) are searched is reduced one ply if 1. The side moving is not in check 2. The move being made is not check 3. No extensions have been triggered for this node 4. No other reduction have kicked in If the reduced search returns a score better than the best score found so far then the move is re-searched using the normal depth. 05-APR-04 --------- S o Changed the move ordering of the main search to the following: 1. Hash move 2. Killer mates 3. Queen-only promotions (both w/ and w/o captures) 4. Winning/even captures 5. Non-mate killers 6. Castling 7. Non-losing mundanes 8. Losing captures 9. Losing mundanes 10. Minor piece promotions (w/ and w/o captures) 04-APR-04 --------- S o Modified the SEE function so that it handles non-captures. With this modification non-captures (or mundanes) are sorted so that losing moves (i.e. essentially, moves that don't place a piece en'prise) are searched last. I o Modified the commands "swap" and "swap-all" so that they now return the SEE value for non-captures as well as captures. 02-APR-04 --------- S o Major revision to the move generator functions in anticipation of some move ordering experiments. The move generators now have much finer granularity (e.g. pawn promotions to a queen are separate from pawn promotions to a lesser piece, castling moves are broken out etc). 31-MAR-04 --------- S o Modified the search so that on a fail-low at the root the search is restarted. According to Schaeffer and Plaat, fail-lows are resolved faster using this method (sometimes dramatically) since the hash-table is already seeded with moves from the previous search. 29-MAR-04 --------- E o Modified the "eval-test" command so that *all* evaluation terms relating to kingside or queenside castling are ignored. With this change the evaluation-only test will (if correct) give the same score no matter how the position is flipped or mirrored. 28-MAR-04 --------- E o Moved the code that gives a bonus for rooks behind passed pawns out of the rook evaluation code and into the (called much less) passed pawn code. 21-MAR-04 --------- I o The code used to run the "test-html" command was cleaned up. I also created a new pseudo-print statement (printT) that prints the test output information to stdout, the log file and the html output files (if enabled) all simultaneously. 18-MAR-04 --------- S o Modified history table accesses so that they now take into account the type of piece moving as well as the origin/target square information. So far, this seems to be an improvment, but as always more testing should be done. 16-MAR-04 --------- E o Evaluation tweaks and bug fixes. Alterations included: 1) Rooks behind passed pawns are scaled up depending on the amount of enemy material. Also the overall base bonus was reduced since it was getting too large. 2) The "trade-off-when-ahead-in-material" bonus was scaled down. 3) The bonus/penalty for king placement is scaled up depending on the amount of enemy material. 4) Corrected an asymmetry bug in the code that flags wrong color bishop/rook pawn draws. 12-MAR-04 --------- O o Myriad first pass changes with the ultimate goal of making the program 64-bit clean. Some of the changes, a new 32-bit unsigned integer type (uint32_t), 64-bit inline assembly code for the functions "findlsb/findmsb", changes to the print() routines etc. 11-MAR-04 (ver. 0.849) ---------------------- B o Bug fix that could possibly cause a crash in extremely deep searches. This showed up mainly in the endgame when the engine was allowed to ponder for an inordinate amount of time. 20-FEB-04 (ver. 0.848) ---------------------- S o Thanks to a tip from Tord Romstad, Djinn now only probes the endgame tablebases if the remaining search depth is 4 plies or further away from the quiescence search horizon. 16-FEB-04 --------- S o Started reworking the time management routines. The first change, the program allocates more time for itself if it is pondering. Also the reserve "safety" margin value was reduced so the program will give itself more time. This bears watching but it seems better. 12-FEB-04 --------- I o Modified the various print routines using a suggestion posted by Lance Perkins on the Winboard forum that eliminated the static print buffer (which is a savings of 4k!) and made the code simpler. 08-FEB-04 --------- B o Fixed a bug in the KPK rook pawn evalution code. The KPK eval code was returning an incorrect winning score if the pawn was a rook's and the enemy king was able to control the queening square (it's a draw since the king can never be driven out). 07-FEB-04 --------- S o Implemented a (reduced) form of Rebel's "max_checks" idea in the quiescence search. The program now sees some tactical ideas a bit faster without any substantial slowdown in the normal search. S o Implemented verified null-move pruning for endgame positions. I o Added a number of CHANT related enhancements for displaying the game tree. I o Added the "pov-display" option to the configuration file (requested by Lars Hallerstrom). This option allows the user to determine the point-of-view of the score displayed by the program. There are four possible settings, "white/black/engine/player". 04-FEB-04 (ver. 0.840) ---------------------- I o Added a fix for the Arena SAN representation bug. Occasionally, Arena will incorrectly disambiguate a move and send the resultant illegal SAN move to the engine. Djinn would reject this illegal move and wait for the correct move, which would never come- ultimately causing the engine to lose on time. 03-FEB-04 --------- E o Tweaks and changes to the pawn evaluation code. Specifically, it pays more attention to candidate passed pawns, hidden passed pawns and 3:2/2:1 pawn majorities. 30-JAN-04 --------- I o Last minute ICC interface changes for CCT-6. The program no longer automatically sends out a "seek 15 3" command after every game. Instead it now relies on the "-zippyGameEnd" command to continually challenge players on ICS. This is more flexible and probably more inline with how other engines do it. 19-JAN-04 --------- S o Changes to the quiescence search and futility pruning, (the pruning is applied *less* aggressively in some instances). 15-JAN-04 --------- E o Revamped a number of the king safety features. Specifically, some of the king-pawn pattern routines were enhanced to flag "danger" patterns to the king. 11-JAN-04 --------- S o Altered the "gen_[white|black]_check_replys()" functions so that they only return legal moves when the king is in check, instead of pseudo-legal moves. This simplifies the one move extension. S o If we are in check and have only two legal replies the search is extended. The extension is half of the one reply extension. S o Double-checks are now identified and extended. I o Minor format change, affecting only the Windows version, for the output of the "perft" command. S o Minor null search optimization. 10-JAN-04 --------- I o The engine now sends the "feature done=0" command before it does anything else (even before it receives the "xboard" command or the "protover" command from xboard). This change will force xboard, and xboard-compatible GUI, to allow more time for initialization (hopefully, this will allow the engine to run under the Wildcat GUI). I o Added the "swap" and "swap-all" commands to allow testing of the static-exchange evaluators at the command-line. 26-DEC-03 (ver. 0.828) ---------------------- B o (Hopefully!) corrected the crashes people were seeing when they inadvertently had an incorrect log file path OR told the program not to produce a log file at all, via the configuration file. Encountering either of the two aforementioned conditions would cause the program to crash at the end of a game. 23-DEC-03 --------- B o Caught a *major* bug in the usage of the scaling arrays. Positional element scores (passed pawn bonuses etc.) were being scaled *backwards* (i.e. as material came off the bonus for passed pawns was DECREASED!!). This affected a number of evaluation elements (ouch!?- how does this thing *ever* play a legal game of chess??) 22-DEC-03 --------- O o The "Scale1to2, 1to3, etc." arrays were expanded from 8 entried to 32 entries. In addition to making the arrays more accurate this change also negates the requirement of normalizing the index (which should speed things up somewhat). S o The history tables no longer incremented using "+= ply*ply" but instead use "+= ply". This provided slightly better move ordering when tested against a number of test suites. E o The positional portion of the evaluation is scaled upward if we have a piece imbalance (pawns don't count). In these type of situations it is important to give more weight to the positional elements to really understand what is happening. S o Implemented the first pass of Ernst A. Heinz's Interior-Node Recognizers. Currently, only a few recognizers are implemented, (KK, KBK, etc.). But this has the *potential* to have a HUGE positive impact on the search (especially in the endgame). This is also the first step to implementing bitbases (ala' Yace). E o If we are in an opposite bishop endgame the score is scaled towards zero. This was noticed by Tord Romstad in a game against Gothmog. The knowledge was there previously but was not being used correctly. 20-DEC-03 --------- O o Reduced the size of a hashtable entry from 24-bytes to 16-bytes with no loss of accuracy or information (it does require additional bit-twiddling). Also a general cleaning up of the hash table routines to make them (slightly) more readable. B o Fixed a minor bug in the call to the function setting the exponent (and thus the size) of the pawn hash table. One of the parameters to this routine is the size of a single pawn table slot, unfortunately, it was being passed the size of a single "hashslot_t" entry which was incorrect. Hopefully, this should fix part of the problem Olivier was seeing when (re)setting the pawn hash table size. O o Reduced the size of the pawn data stored in the pawn hash table to 24 bytes, so that 24 + 8 bytes of the hash key = 32 bytes. O o Removed the right shift of 1 from the diagonal and file rotated attack routines. It was not eliminated from the rank attack since the "occ_rr00" map is used by the evaluation function. 14-DEC-03 --------- E o Modified the code that rewards the giving up of the outside passed pawn. This code is used to transistion into an ending where the winning king gobbles up the offside pawns. 07-DEC-03 --------- I o Changed the precedence of determining where the configuration file is located. The program now looks for it in descending order (this change makes it easier to test experimental versions): 1) The current working directory 2) The "DJINN_RC_PATH" is used if set 3) The user's home directory 29-NOV-03 (ver. 0.815) ---------------------- S o The value of one ply is now equal to 64 instead of 16. Using a larger value for a single ply allows the half and quarter ply extensions to scale more smoothly. 28-NOV-03 --------- S o The draft is now calculated using the fractional "ply" value instead of the "depth" value (the draft variable is now an integer, instead of a byte value). S o The extensions are now tapered off as search moves deeper. I o Added more user defined CHANT node types. 27-NOV-03 (Thanksgiving!!) -------------------------- B o Fixed a bug in the bishop evaluation code. Bishop attacks against the enemy king were not being counted in the king safety code. E o Converted the two-dimensional material-left "Scale[][]" arrays into one-dimensional arrays which use the _enemy_ material to set their values. In reality, it's not that passed pawns become more valuable as material comes off the board, it's that *my* passed pawns become more valuable as *ENEMY* material comes off the board. 24-NOV-03 --------- S o Added support for Andrei Fortuna's excellent CHANT library. Deep search trees can now be printed to a database file and browsed graphically (very cool!). 26-OCT-03 (ver. 0.808) ---------------------- E o Changed the scoring of attacks against the white/black kings. E o A king wandering around past the 2nd rank during the opening or middlegame *and* having no safe squares to move to is heavily penalized since he is likely under heavy attack. E o The STable[] attack array has been modified to account for two minor pieces attacking the king vs. two pawns (two minor pieces represent more danger and thus should be given precedence). E o Increased the two bishops bonus. S o The transition into a pawn and king/pawn-only-endgame is now extended an extra ply. E o Changed the way the C (or F) pawn being blocked in a queen's pawn game is accessed. 12-OCT-03 --------- E o Removed the king stuck in the middle penalty since it seemed too arbitrary. E o Converted the undeveloped pieces penalty to a response curve instead of a straight multiplier. E o Changed the way the castling bonus is accessed if the king resides on either the D or E files. 11-OCT-03 --------- O o Regressed the "gen_[white|black]_cap" function to the previous non-LVA/MVV version. This version is *much* faster and is ultimately a better tradeoff. 05-OCT-03 --------- B o Rolled in the bugfix for the locking up after a match bug that Leo discovered in his WBEC tournament. Is the problem fixed, good question?!?! :-( 01-OCT-03 --------- O o Added a new field to the pawn hash table that keeps track of *all* the files which contain a white or black pawn. This is stored as an 8-bit value that can be used to index into a variety of tables. E o In single-bishop vs. single-knight endgames, the side with the bishop is given a bonus if there are pawns on both sides of the board and the center is open. The bonus is scaled depending on how many files are between the pawns. E o Both bishops are now folded into the attack table around the kings. 29-SEP-03 --------- E o In the endgame the kings are now given a bonus for being in front of any passed pawns. B o Fixed a fairly serious bug in the "in_endgame()" function. 27-SEP-03 --------- E o Function pointers are now used to determine which king safety/ attack function will be called depending on the king's pawn cover signature. 24-SEP-03 --------- B o Fixed a *MAJOR* bug in the usage of the SEE function when it was being used to cull captures. Essentially, for captures it was not making the enemy piece on the "from" square the first piece to capture, but was instead simply returning a plus or minus score based on if *any* of the enemy piece could profitably capture on the target square. The net effect is that a large number of __losing__ captures were made during the WINNING/EVEN capture phase. 21-SEP-03 --------- E o Changed how pawn evaluations are added to the pawn hash table. B o Fixed a time-management bug. 19-SEP-03 --------- S o Reworked the way the recapture extension is triggered. This resulted in a *very* nice increase in the program's tactical strength (especially, on WAC 235). 17-SEP-03 --------- O o The captures are now generated in MVV/LVA order *before* they are culled using the SEE function. 15-SEP-03 --------- E o More changes to the king safety code. I'm going to need to start playing a series of games to see if these changes have been beneficial or a waste of time. I suspect they will pay off in the long run but I need to tune them so that they make sense. 14-SEP-03 --------- E o Pawn cover defects are now saved to the pawn hash table. In addition, the program now scores *all* possible pawn patterns in the six squares in front of the king. E o Restructured the king safety code. 09-SEP-03 --------- O o Modified the swapoff function to be more cache-friendly. O o Slight optimization to the in_endgame() function. 08-SEP-03 --------- O o Removed the "piecelist[SIDES][PIECES]" bitmaps from the board class. The program now maintains a set of pointers to the "w_king, w_queens, ... b_pawns" bitmaps instead. O o Changed the search so that winning, even and losing captures are all generated at the same time. Previously, all the captures were generated twice, once when we searched the winning/even captures and again when we searched the losers. This was even worse than it sounds because they were sorted twice as well. B o Fixed a bug in the outside passed pawn code. It wasn't handling correctly the case where each side had outside passed pawns (on the same side of the board) that had passed each other. For example, an outside white passed pawn on G5 and an outside black passed pawn on G4 where scored incorrectly. 07-SEP-03 --------- E o Added an evaluation term to penalize trapping a rook in the corner by moving the king towards it. E o Tweaks to the two bishops bonus code. 06-SEP-03 --------- E o Added the code so that the program will recognize a draw if one side has no pawns and the other side has only a rook pawn(s) under the conditions: 1) The attacking king is in *FRONT* of its rook pawn 2) The defending king is directly lateral to the attacking king such that he keeps the other king from ever getting off the rook's file. E o Toned down the program's willingness to sacrifice material for an attack on the enemy king. Sometimes it was spectacular, but other times it was just embarassing ;o B o Fixed a bug in the A7/H7 or A2/H2 trapped bishop code. The program was missing instances when exchanging the bishop for the two pawns was a good trade if it ripped the king's protection apart. 05-SEP-03 --------- I o Enhanced the "eval" command so that it actually displays enough information to be useful ;) E o Fixed a bug in the half-open rook determination code. 02-SEP-03 --------- S o *Major* rewrite of the repetition hash table code. It should consume half the memory it previously used and be more accurate. 01-SEP-03 (build: 770) ---------------------- E o Tweak to the opposite castling code. Duos in the same quadrant as the enemy king are given a bonus. E o Added the first pass at real outside passed pawn code. B o Fixed a bug in the repetition hash table initialization code. 31-AUG-03 --------- I o Added a large amount of extra PGN support. EGTB moves are now annotated as such when the program writes the game out in PGN format *and* the verbose switch is on (the default). It also now writes out the PGN header info. I also intend to add a command-line switch for writing the PGN saved games to a separate file in addition to the log file. 30-AUG-03 --------- S o Tweaks to the search. Mate killers are now used in the qsearch. E o Minor tweaks to the knight evaluation code. E o Added penalty code for hanging pieces and pieces pinned to hanging pieces. E o Fixed a bug in the code that is called in king/pawn-only endgames to evaluate the position when only one side has any passed pawns. It was declaring the position won even when black had enough counter-play to at least draw. 27-AUG-03 --------- I o Added a 16-byte signature check to the beginning of the opening book. This signature will be automatically generated whenever a new set of random numbers are generated. I thought about making this the MD5 signature of the random numbers but that seemed like overkill. 24-AUG-O3 --------- E o Added a first pass at the king offside penalty. This penalty kicks in when we are in a king-and-pawn-only endgame with all of the pawns on one side of the board. If one of the kings is on the opposite side of the board (away from the action) then he is penalized. This can happen if he was lured away by an outside passed pawn. 23-AUG-03 --------- S o Pawn hash moves are also copied now and not dynamically updated. S o Modified how often the "egtb_pv" function (used to update the PV when we have an endgame table hit) is called during the search. Endgame hits are treated more like endpoint mate positions and not extended. This is faster and allows the endgame to be accessed deeper in the search. S o Increased the accuracy of the futility-cutoff in the qsearch. 22-AUG-03 --------- I o Added the "-xlog" command-line option so that we can control the generation of a log file in a much more flexible fashion. 21-AUG-03 --------- B o Fixed a minor bug in the endgame tablebases code. The program was erroneously saving a move to the hash table even when the last move played delivered mate (a definite no-no). 20-AUG-03 --------- B o Fixed a bug in the "board_t" == operator. It was comparing the location of the black king to the location of the other board's white king (and obviously failing). O o The "hash_move()" function is no longer used to undo a hash move. Instead before we make any moves at a node we save the hash value and after we takeback a move we simply copy the old value instead of computing it using the Zobrist technique. 19-AUG-03 --------- E o Minor eval tweaks. I added a number of new DEBUG EVAL macros that will make future changes to the evaluation function much easier and simultaneously more accurate. B o ???? - Disabled lazy eval. It *may* have been completely screwing the program's evaluation. Shroder has warned of this numerous times and it may have bitten me!! 18-AUG-03 --------- E o Revamped some of the rook scoring code (is it better well, we shall see :) 17-AUG-03 --------- E o Implemented Bahr's Rule. E o Improved the accuracy of the wrong-color bishop with a single rook's pawn. This solves the cute little stalemate problem Hyatt posted to CCC awhile back. 16-AUG-03 --------- E o Added a penalty for an isolated D-pawn. Since the engine really doesn't understand either the advantages or disadvantages of the IQP, it becomes a liability 9 times out of 10. 14-AUG-03 --------- E o The "attack_index" into the STable[] is added to the king safety defect count for each side. o Renamed FilesMask[256] to Files255[256] and renamed IslandsMask[8] to FileMask[8]. E o Increased the QUEEN_IS_STRONG bonus from 45 to 125. O o Removed the incremental function "update_castling_penalty()" from the makemove() function. Instead the program takes advantage of the new castling data structure to compute the castling penalty on the fly in the evaluation function, when needed. O o Removed both the "game_phase" field and the "castling rights" field from the evaluation hash table (they are no longer necessary). o Increased the Hamming distance of the random numbers used by the hash tables. This new set has a Hamming distance of 24 minimum, (the pawn hash table number is 27). B o Fixed a bug in the way the castling penalty was being accessed. Essentially, the penalty was being set too large because it was being accessed multiple times (if for example, the queen rook moved and later the king moved, a penalty was accessed for both moves when it should only have been given for the king move). 09-AUG-03 --------- O o *Major* changes to the rotated bitmaps code. The changes have increased the speed of the program by 20-25%, with no downside (except for my blood, sweat and tears :) 05-AUG-03 (ver. 0.739) ---------------------- O o Tweaked the rotated-bitmap code so that the arrays are smaller. Previously, they were bitmap_t rrb[64][256], now they are rrb[64][64]. The change gave roughly a 5.4% increase on the PIV and a little over 3% on the Athlon. This came about because of a thread on CCC between Gerd and Foden. A number of additional space optimizations are possible and in the pipeline. 31-JUL-03 --------- B o Adjustement to the conventional time search (e.g. 40 moves in 5 min. etc.), thanks to Leo Dijksman for pointing it out. B o Corrected a second time management bug in the pondering code. When the program was switching from a ponder search to a regular search the time for the program's move was not getting recalculated. I also added in the 15 sec and 8 sec panic modes that the incremental search uses. S o Eliminated the separate replace-always hash table. Now the 3rd hash slot is a replace-always slot. This should reduce the cache misses when accessing the hash table. S o Tweaks to when the endgame tablebase gets probed. 27-JUL-03 --------- o Release Candidate 1 (RC1) 26-JUL-03 --------- I o Added configuration file support (either .djinnrc or djinn.rc). B o BoundsChecker, Valgrind and PC-Lint clean. I o Added the "-xbook" and "-xegtb" command-line options. These options can be used to disable the use of a book or endgame tablebases, even if paths are entered for them on the command- line or in the configuration file. I o Added an option to set the log output directory. This can be set in both the configuration file or with the environment variable DJINN_LOG_PATH. 16-JUL-03 --------- I o Revamped and enhanced the cpu timing routines for both Windoze and Linux. In fact, a redo of the major "get_time_??" functions. 15-JUL-03 --------- B o Fixed a bug that was cropping up when the program was involved in a normal search (the problem was not showing up during analysis). Essentially, the engine was delaying the processing of the "quit" command until *after* the search returned (not good :( 14-JUL-03 --------- B o More work in preparation of the first (limited) release to another human being. Whenever a move was undone the size of the positions vector was *not* being reduced, which meant that the best move was being incorrectly flagged as a repetition- and hence avoided. The fix was to use the "Positions[]" vector index instead of the .size() operator. 13-JUL-03 --------- I o Mega-push to release a Windows version of the program. A large amount of clean-up and work. Not done testing yet, nor debugging but it is very promising so far. 30-APR-03 --------- S o Reduced the "contempt" factor to a more reasonable range. Also if we enter a king and pawn only endgame then we set the contempt factor to zero if it is less than zero, since it is not a good idea to enter a king and pawn only endgame with a negative contempt factor. 28-APR-03 (beat Gnuchess 4.0pl80 at 15 3, +44 -34 =22) ------------------------------------------------------ S o Added a modulo-8 age counter to all of the main hash table entries. This is a 3-bit counter that gets incremented each time we start the first search for a new move. This count gets stored in the "age" field of an individual hash entry to flag it as a current move. The advantage is that we don't need to age the hash table before searching which could screw up all the data in the cache. o The 28-APR-03 version was the first version to beat GnuChess 4.0pl80 in a 100 game match (time control 15 3). The final score was +44 -34 =22. 27-APR-03 --------- E o Scaled the protected and duo passed pawn scores back slightly, since they were getting too large. E o Altered the isolani penalty so that passed pawns are not penalized. E o A knight which is immune from attack by enemy pawns, is now given a bonus if it blockades doubled pawns, (previously, the bonus was only given for blockading passed pawns). S o The pawn, evaluation and main hash tables are only cleared when the root position is loaded (which means it is *not* cleared between moves). 26-APR-03 --------- E o Pawn structures values are only scaled from 2:1 as the game progresses. E o Added a non-linear isolani penalty. 21-APR-03 --------- E o Passed pawn values are now scaled from 4:1 based on the amount of material (not counting pawns) left on the board. This is accomplished via the "ScaleXto1" arrays which provides a smooth scaling factor as the game progresses. 20-APR-03 --------- B o Fixed a bug in the way the trapped bishop penalty was being scored. B o Fixed a bug that was in the bishop and passed rook pawn code, (it was assigning the "RIGHT_BISHOP_BONUS" incorrectly). E o The pawn structure scores are now scaled from 3:1 using a multi- dimensional 8x8 array based on the amount of material present. E o The number of pawns for each side is no longer stored in the "eq_pawn_material[WHITE|BLACK]" variables. The program always subtracted them out before using them anyway :( 19-APR-03 --------- I o Added the "flip", "flop" and "flip-flop" commands for mirroring the current position around the x, y and x/y axis. I o Added the "eval-test" command to run an epd test suite but only score the positions using the static evaluation function. Note, that each position is scored four ways: 1) normal 2) flipped about the x-axis 3) flipped about the y-axis 4) flipped about both axes. Any position that has a score that deviates as the position is translated is flagged. B o Fixed a bug in the rook evaluation routines uncovered by the mirror code added in the above bullets. The program thought it had a rook behind every passed pawn on the board?!?! Very bad :( B o Fixed a bug in the king safety code, also uncovered by the mirror changes. B o Fixed a bug in the bishop scoring routines for the fianchetto bonus. The black fianchetto bonus was being increased if the white queen was near the enemy king, when it should have been increased if the black queen was near the enemy queen. B o Fixed another *very* serious bug in the king safety scaling code. If the black king's safety was marginal then it was incorrectly being set to the scaled *white* king safety value?! B o Fixed a bug in the black candidate passer pawn code. 17-APR-03 --------- I o Added a new function to support the "eval pawns" callback routine. E o Added an additional check so that the program will not castle into unsafe pawn protection if the other side of the board has a better (i.e. safer) pawn shield. 13-APR-03 --------- E o Tweaked king safety so that the program will be much more reluctant to have its kingside ripped apart by a piece sacrifice. 12-APR-03 --------- B o Fixed a bug in the "evaluate_black_passed_pawns()" code. The 2nd call to swapoff() to determine who controlled the square in front of the black passed pawns was using the wrong side to move. E o Added code to recognize "hidden" passed pawns on the 6th rank. 06-APR-03 (*Bug fixes only*) ----------------------------- B o In the new time management code the value of the original time value is cast to a "double" before it is multiplied by the fail-low scalar. B o Valgrind caught the fact, that I wasn't releasing the memory used by the "avmv" member when running a test suite. B o In the Gaussian-Jordan elimination code, the "irow" and "icol" variables are set to -1 and checked before being used. B o The "Game" object method ".pop_back()" verifies that the game list is *not* empty before trying to undo a move. B o Fixed a bug in the "evaluate_black_passed_pawns()" code. The 2nd call to swapoff() to determine who controlled the square in front of the black passed pawns was using the wrong side to move. 01-APR-03 (ICC rating: 2388) ---------------------------- o Fixed a time management bug. The "update_[programs|opponents]_clock" function was comparing the program/opponent's time to <= zero, even though the variables are *unsigned*!! The result was that when the program was close to running out of time the time/move variable would be set to an astronomically large value (1.xxe+06), which of course caused the program to lose on time. o Changed the "Game" object so that moves are added or deleted in three places only 1) make_players_move 2) make_computers_move and 3) retract move. 28-MAR-03 --------- o Fixed another ponder bug. If I ever get ambitious, the state-machine code should really be rewritten, since it is by far IMHO, the buggiest code in the program (certainly the most obtuse). 27-MAR-03 --------- o Changed the enumeration commands to "perft" commands to match the rest of the chess programming world. I also changed their structure some- what to provide additional information. o Fixed a bug in the "Game" object saving of a move when in "analysis" mode, that caused a hard crash. 26-MAR-03 --------- o Fixed a bug that would only allow the engine to ponder if the opening book was loaded?!? o Fixed an additional ponder bug that caused the engine to go into the wrong state when it found a mate *and* the user selected the ponder move. Since it was in the wrong state it would never move and would lose on time. 23-MAR-03 --------- o All fail-highs (both black and white) are now shown with a ++. While all fail-lows are shown with a --. This is consistent with what everyone else does and in addition, seems more clear. o Tweaks to the king safety routines and to the queen placement table. In addition, the rook immobilization code is commented out for the moment. 22-MAR-03 --------- o The program now prints out a histogram of EPD test results. o The search time is no longer extended on a fail-low when we are running a test suite. o The time management of a fail-low failure has been modified so that the engine will extend the time *proportionally* to the size of the fail-low value. This is capped at a maximum of 5x the originally allocated time. Also all fail-lows of a knight or greater are given the max 5x multiplier. 19-MAR-03 --------- o The recapture extension is skipped if we are in the losing captures phase of the search. o Underpromotions are now completely skipped in the qsearch. 18-MAR-03 (tied Gnuchess 4.0pl80 at 15 3, +38 -38 =24) ------------------------------------------------------ o Fixed a rather serious bug in the endgame evaluation tradeoff routine. The check was coming *after* the king-pawn-only early exit, which is fine until there are no pieces left. With all the pieces exchanged off the tradeoff routine was never getting called :( The end result was that the program would trade off everything *except* the last piece, (which is exactly what you don't want to happen). o The 16-MAR-2003 version was the first version to tie Gnuchess 4.0pl80 in a 100 game match (time control was 15 3). The final score was 38-38-24!! 16-MAR-03 --------- o Changed the scaling of the passed pawns so that 5 and 6 total pieces will also slightly increase the value of the passed pawns. o Added a "evaluate_king_safety" wrapper function that is used to pull enemy pieces towards the king which is less safe. 15-MAR-03 --------- o Removed bishop mobility, immobility and piece table scoring. These terms were causing the program to play some odd pawn moves. o Revamped the king safety code so that the program "hates" an open B-file or G-file to the king. In addition, there I made a number of general king safety tweaks. o Alter the knight evaluation routines so that bonuses are handed out for multiple items (such as a knight immune from enemy pawn attacks, etc.) along the way instead of just at the end of the routine. o The castling bonus is now treated like any constant evaluation term. It is negated (or not) in the evaluation function depending on the color of the side moving. This simplifies the makemove function and clarifies the various evaluation routines. o Added a DEBUG_PAWN_HASH makefile switch. o Scaled back the passed pawn scores unless there are 4 or less *total* pieces on the board. o Pulled the scoring of space out of the evaluation function. 13-MAR-03 --------- o Removed all the evaluation constants in "eval.h" that have become obsolete. This makes the file *much* easier to read and modify. o Added some knight evaluation terms that got removed along the way. o Removed some pawn scoring routines that were both buggy *and* never used. They were just cluttering things up. 11-MAR-03 --------- o Pawns are not flagged as weak if they are bracketed by two same- color pawns. o Altered the entombment code of the bishop to match the way it is done for the queen and rooks. 10-MAR-03 --------- o The entombment check for the queen is skipped in the opening. Otherwise, it could encourage the queen to run around before the minor pieces are developed. o Fixed a bug in the way the castling bonus was being assigned. 06-MAR-03 --------- o Increased the new king safety table "STable" from 64 entries to 256 entries. It now handles multiple pieces of the same type being near the king. 02-MAR-03 --------- o Changed the EGTB mode after a discussion I had with Bob Hyatt and Dieter Buerssner (of Yace fame) on CCC. Tablebase cutoffs are no longer allowed at the root. I also initialize the PV moves to the "EmptyMove" before updating the PV. This means that if stalemates are stored in the hash table they will have the "EmptyMove" stored as their hash move, which is OK. o Stalemates are now stored in the hash table during the regular search. The "EmptyMove" is stored as the hash move (the same technique is used for actual mate postions that are stored). o Added a MATE_PV marker flag to principal variation moves. o The moves that are extracted from the tablebases when the principal variation is updated (in egtb_update_pv) are stored in the hash table. This prevents the problem of the PV getting truncated. 01-MAR-03 --------- o Fixed two bugs in the egtb code. (1) At the end of a mate variation there is no move to store in the hash table or load into the PV, so skip it. (2) When a stalemate is found, the same criteria apply, so again- skip updating the hash table and principal variation. 28-FEB-03 --------- o Fixed an en'passant bug caused by all the data structure changes lately. The en'passant target square was not being translated before it was being passed to Nalimov's EGTB routines. This was caught by Nalimov's egtb routines when run in debug mode! o Widened all the variables that hold the enumeration counts to 64-bits. The previously used thirty-two bits, would roll over too easily. 27-FEB-03 --------- o Added debug code to display position objects. Removed the board_t to position_t operator since it was confusing because of the fact that a board object contains "state" (i.e. depth) information. Instead an explicit function called "convert_to_position()" is now called to make the conversion which requires the depth to be explicitly included in the function call, to avoid confusion. 25-FEB-03 --------- o Added recognition in the endgame, of the fact that many types of pawnless endings where one side is up by a minor piece (for example, R+N == R, R+B == R etc.) are a draw. Hopefully, this will keep the program from getting swindled in won endgame positions. o Fixed a minor bug in the way the en'passant square was being used in the FEN translation code. 24-FEB-03 --------- o Added queen immobility checking, similar in nature to what we do for rooks. This should help keep the queen from getting trapped. o Added the first cut at a major revision of the king safety. Using a number of ideas from Ed Shroder's web page. 23-FEB-03 --------- o Moved all the location bitmaps (rotated and non-rotated) into a single structure, so that accesses to the four bitmaps will be sequential and hence more cache friendly. o I tweaked the value of the pieces slightly so that the program would be less likely to make bad trades. o Fixed a bug in the castling_rights_gone() function. o Fixed a bug in the search- the KILLER_MATE_PASS was not getting the same checks as the KILLERS_PASS (specifically, the special moves pass was not being skipped). 22-FEB-03 --------- o Added en'passant captures info into the hash signature. o Fixed a bug in the eval function that was making illegal moves. Actually, the moves were legal except they were legal for the other side, which did *not* have the right to move :) 20-FEB-03 --------- o Discovered two subtle (but very real) bugs while checking out the new hash stuff. 1) When making a NULL move the castling rights were *not* being copied to the next ply in the search. Essentially, both sides were losing the right to castle after making a null move!! 2) The fifty- move counter was not being incremented across hash moves, also invalid, but more than likely, less serious. 19-FEB-03 --------- o Added castling rights to the hash signature. This also entailed restructuring the way castling rights are stored (the new structure is simpler). 16-FEB-03 --------- o When analyzing, the program now displays the available book moves and how often they were played (the information is displayed as a percentage) in the xboard analysis window. o Added a PGN class for handling all PGN operations by the program. This needs substantial work, but it's a good start. o The "termination" signal from xboard is ignored, instead the program shuts down on its own instead of being sledgehammered to death by the OS. 15-FEB-03 --------- o Added the "book-style " command. This affects the way the program will select a move from the opening book. --------------------------------------------------------------------- solid - No move will be played from the opening book that was played less than 25% of the time. This is the default. freq - This causes the program to select a random move that is weighted towards the move played most frequently in the the opening book for the current position. random - Will cause the program to play a truly random move. All moves are given *equal* weight. first - Causes the program to always play the move that was played the most often in this position. --------------------------------------------------------------------- If a move is followed by a '!' then it is considered best and the program will always play it (regardless of the style setting). Conversely, if a move is followed by a '?' it is considered a blunder and will not even be included in the book file. 14-FEB-03 --------- o Major restructuring of the opening book and the supporting data structures in preparation for book learning. o Added the "write-book [--verbose|-v] " command, which writes the currently loaded opening book to an ASCII file. This includes the frequency of the move (i.e. how often it was played in this position) as well as wheter the move was a blunder or brilliancy. 09-FEB-03 --------- o Implemented a "tree_t" class to encapsulate all of the tree logging routines. Hopefully, this will make them easier to manage, use and extend for *all* classes, not just the "engine_t" class. o Split out promotion moves as a separate pass in the normal search. Non-queen promotion searches are reduced by one ply. 08-FEB-03 --------- o Added Ed Shroder's idea of "killer mate" moves. This is now the 2nd pass tried after the hash move. It seems to give a nice speed up to a number of mate positions (WAC141 gets solved in roughly 3 seconds instead of 17 seconds, among others). It also found four additional WAC positions as mates. o Fixed a bug in the fixup_pv() that was corrupting the main hash table. The initial hash move that was being passed to the fixup PV routine was being passed as a "reference" for some unknown arcane reason. The hash move was being corrupted inside the routine so that it was no longer valid for its accompanying hash key. o Fixed a *major* bug in the way the qsearch was handling the hash table. Previously, for some odd reason, I was under the mistaken impression that the draft didn't matter for the qsearch. WRONG- it matters a great deal!! I elminated the separate qsearch "qmatch()" function and simply replaced it with the "match()" function. This now solves the "mate9.fen" position in just a few seconds!! 06-FEB-03 --------- o Changed the advancement values for a passed pawn, a protected passed pawn and passed pawn duos. 04-FEB-03 --------- o Added the ADD_EVAL() macro to the board.h file for debugging pawn evaluation scores. o Added "candidate" passed pawns to the passed pawn evaluation code. o Fixed a serious sign bug in the passed pawns code for black as well as a major overhaul of the "weak" pawn code. o Numerous pawn evaluation tweaks and a large amount of weight tuning. 02-FEB-03 --------- o Added an initial cut at asymmetric king safety scoring. 01-FEB-03 --------- o Reworked the move_t class so that a move is stored in an unsigned integer instead of a byte + short int combination. Hopefully, this will be faster. o Made futiltity depth reduction a compile time option (off by default). o A large number of speed tweaks. 27-JAN-03 --------- o Fixed a xboard bug. The program was disambiguating the move after it was made on the board, which sometimes gave an incorrect result. Xboard flagged it correctly as an illegal move. o Added futility-pruning at the normal search horizon (the ply just before we start into the quiescent search). Only "mundane" moves that are not a promotion are candidates for being pruned. o Null-move beta cutoffs are now hashed. o The 50-move draw counter is now incremented/decremented when we make a null move. 26-JAN-03 --------- o Complete rewrite of the "rule-of-the-square" code. o Added a check in the evaluation function for stalemates of the lone king. 23-JAN-03 --------- o Evaluation tweaks. Mostly, to the queen evaluation routines. 21-JAN-03 --------- o Added a mobility/entombment check to the rook evaluation functions. 20-JAN-03 --------- o Changed "ply" so that it is decremented instead of being incremented. The variable "depth" and "ply" now move in opposite directions. 19-JAN-03 --------- o Fixed some evaluation bugs (unfortunately, this seems to *always* be fertile ground). o Added a first pass attempt at adding a "cramping" bonus. 18-JAN-03 --------- o Added a Flip[] array that flips the black piece table values when they are initialized in the Oracle function. This make it simplier and less error prone since only the white tables have to be maintained by hand. o Increased the piece table penalty for putting a knight and queen on edge of the board. o The piecetable bonus for the queen (and knight if a queen is on board) is increased for the squares directly surrounding the enemy king. 17-JAN-03 --------- o Fixed the 6th rank duo and connected passed pawns code. This solves WAC 2 quicker than any other code I've tried and produces a better principal variation. 15-JAN-03 --------- o Added the TREEQ define and pulled the explicit "depth" variable out of all the TREE? calls. Instead both "depth" and "ply" are implicitly passed to the TREE calls. o Changed the draft of EGTB positions stored in the hash table to DRAFT_INFINITY. o The EGTB is not probed unless the move made at the last ply was either a capture or promotion. Also I no longer define the 4-1 tablebases, since the side with the four pieces versus the lone king is almost always a win or loss for somebody. The default EGTB cache is now 8M. All these changes results in about a 3-4x speed up in the engine when it now hits the tablebases. 13-JAN-03 --------- o Added "NOISE_DEPTH" to the engine_t class. 12-JAN-03 --------- o Added the "determine_cpu()" function back into the engine initialization routine. The mask interrupt value is decreased (i.e the search is interrupted more often) if the time left in the game starts to get dangerously close to zero. o Cleaned up the structure of the set_move_search_time() function. 07-JAN-03 --------- o Added an inline [white|black]_rank() function to get rid of the 7-rank statements peppered throughout the code. 03-JAN-03 --------- o Fixed some of the king safety evaluation code that was incorrectly converting between floating point and integer values (the values were incorrectly being zeroed out in some cases). o Changed the way the enemy piece tropism is scored in the king safety routine. o Changed the way the fianchetto pawn structure is scored. o Added the "pawns on the same color as a single bishop" heuristic back into the bishop evaluation code. 02-JAN-03 --------- o Added the bad bishop code back into the evaluation routine with a bit of a twist. The center pawns that trigger the bad bishop penalty must be protected by a friendly pawn for the code to kick in. o Added the bishop fianchetto bonus/penalty code back in. 01-JAN-03 --------- o Fixed a bug (or two) in the knight evaluation code. o Fixed a bug in the Oracle knight pre-evaluation. 22-DEC-02 o Fixed the "myname" command to xboard. 07-DEC-02 o Tweaked the knight evaluation routines. o Altered the way the passed pawn scores are scored. The passed pawn scores are only scaled if the side with the better pawn score has at least as many pieces as the other side. Hopefully, this will keep the program from giving away pieces for passed pawns (a dubious practice at best). 01-DEC-02 o Tweaked the king safety castling patterns. o Modified the piece tropism terms for the enemy pieces near the king. o Added the pinned piece code back into the endgame evaluator. o Modified the findmsb/findlsb inline assembly code. o Reintroduced a modified (and hopefully more accurate) version of the two-connected passed pawns versus a single enemy piece in the endgame. 19-NOV-02 o Slowly phasing evaluation term back into the program in an effort to weed out junk or conflicting terms. 13-NOV-02 o Started using Valgrind and it found a *major* bug in the hash table function "match()". The value of the bound was occasionally being used even though it hadn't been initialized. Fixing this bug seems to have doubled the nodes per second!!! Can this be correct?! 09-NOV-02 o Added code to flag passed pawns separated by more than three files as a win (the king will not be able to stop one of the pawns from queening). o Changed the way captures are sorted in the quiescence search. Losing captures are no longer tried at all and winning captures (as determined by the SEE function) are sorted from best to worst. o Fixed a bug in the way the principal variation was being displayed. Occasionally, a move was being displayed in the middle of the principal variation. Now these moves are shown as simple hash moves. 04-NOV-02 o Reworked the code for the square of the pawn so that it is faster, simpler to understand and hopefully more accurate. o Started sorting the captures generated in the quiescence search. 02-NOV-02 o Reduced the contempt factor by two, since it was getting rather large when it would play on ICC. 01-NOV-02 o The best move returned by the previous iteration (after 5 iterations have passed) is searched first at the root. o Added a five entry array that contains pointers to the piece attack functions for the king, queen, rook, bishop and knight. This array is used to simplfy and speed up some of the move generation routine. 24-OCT-02 o Added the "BINARY()" macro for simplfying the setting of 8-bit values. This was an idea stolen from "Game Programming Gems 3", very slick. o Changed the code that assigns the tradeoff bonus. o Moved the updating of the principal variation to the end of the root search and the main negascout search. 23-OCT-02 o Major tweaks to the quiescence search code (this will bear watching). 21-OCT-02 o Reintroduced the single-move extension. o Reintroduced the pawns to the 7th rank extension. o Reduced the initial lazy eval boundary to 1.5 * PAWN_V (it used to be set at two pawns). 19-OCT-02 o Updated the wrong-colored bishop and rook's pawn code to handle doubled rook's pawns correctly. o Fixed a bug in the three lone pawns versus lone king code. It now only accesses the advantage if the pawns are not doubled (they do not need to be connected, but only on separate files). o Reintroduced the "weak" pawn code. 18-OCT-02 o Mundane moves at the root are now sorted based on the scores returned by the static evaluation function. 16-OCT-02 o Changed the way hash probing is done so that hash entries are stored in consecutive locations. This *should* make the hash table more cache friendly. o Reintroduced the recapture extension. Currently, the extensions are *not* limited to one ply per level in the tree. So far this seems to be working OK, but will need to be watched in case the tree starts to explode. 12-OCT-02 o Added code to flip the ideal locations for the rook PieceTables in the endgame if the enemy king has wandered across the board. o Fixed a bug where the rook on the 7th was not being scored correctly. 11-OCT-02 o Added the code so that the program recognizes the draw when one side has a king, rook's pawn and wrong color bishop against a lone king that has made it to the pawn's queening square. 06-OCT-02 o Fixed the "bad trade" code in the evalutation function. 04-OCT-02 o Fixed a bug in analysis mode that was rearing its ugly head when undoing a move. Basically, to undo a move we step back one position- so far OK. Unfortunately, when reloading the repetition hash table the program was loading in the position that was undo, which gave the program the erroneous impression that the best move playable (such as a mate) was a draw. To avoid the draw the program would make another inferior move. 02-OCT-02 o Changed the criteria for declaring a pawn passed. Now the three files in front of the pawn must be clear of enemy pawns AND the path directly in front of the pawn must be clear of *friendly* pawns. 01-OCT-02 o Altered the test suite solutions matching code so that moves returned by fail-low searches do not reset the "solution moves matched" counter. This is reasonable since the program will never play a failed-low move in a real game since they are not reliable. o Checks are only extended if the move being searched is the initial move or the current best score is greater than or equal to alpha. 27-SEP-02 o A *major* revamp in the bitmap_t class. All the constructors were made "explicit" and the "int" operator was removed. In it's place two new functions were created: not_zero/zero. This should prevent the unintended conversion (and comparisions) between integer values and bitmaps. 23-SEP-02 o Changed the way mate depths are reported. Previously, the depth of the mate was being reported as the length of the PV returned. This was incorrect since the PV was often formed out of probes to the hash table, which may be too short. The correct way was to report the depth returned by the search, and display the PV which *could* be shorter than the depth returned if it was truncated. This wasn't a bug, instead it was just very perplexing when I ran mate test suites. 21-SEP-02 o The king piece table values were not being used in the opening and middlegame (now they are- better or worst, well, that's hard to say ;) o Changed the "update_killers()" code so that captures are no longer stored as killers. o Simplified the code that penalizes rooks and queens lined up against the king. It is less convoluted, and thus should be faster than before, and it (hopefully) will cover more cases. o Added the "rook trapped in the corner by the king" code back in. o Caught a fairly serious bug in the "in_the_endgame()" function. The program was comparing w_majors and b_majors as if they were integers instead of bitmaps (oops)??! o I also tweaked the code that the engine uses to determine if it is still in the opening. 18-SEP-02 o Changed the way pawns in the center are scored in the opening. o Only the minor pieces (bishops and knights) are penalized for being undeveloped in the opening. o For en'passant moves and pawn jump moves the piece field is now set to PAWN (previously it was left clear). o Fixed a major bug in the loss of castling rights code. Pawn jumps and en'passant captures were being penalized the *maximum* penalty of roughly 200 points. Who knows how many points this cost when playing Gnuchess. o Also if a rook is captured that could have castled the program gives this the same penalty as if the rook moved. 16-SEP-02 o Mate bounds are now stored in the hash table. They are adjusted so that they are MATE-1000 (or -MATE+1000) etc. Note, for LOWER bound scores the values are adjusted to MATE-1000. For UPPER bounds the adjustment is to -MATE+1000. The other bounds are *not* stored because adjusting the bounds to the other extreme would violate the meaning of UPPER and LOWER (a big bug-ridden no-no). o Also increased the penalty for knights on the rim. o Increased the penalty for undeveloped pieces. 13-SEP-02 o Added a function to determine if we are in an opposite castling type position. This is now called at the end points as well as at the root, which should make it more accurate. o Modified the fianchetto code in the bishop scoring routines so that it gives a bonus instead of a penalty if the enemy has a fianchettoed king but is missing the fianchetto bishop *and* we have the correct squared bishop to threaten the enemy king. Also this bonus is increased if our queen is still on board. o Tweaked the king safety routines. o Changed the way the "evaluate_opposite_castling()" routine looks at open files against the king in those type of positions. 12-SEP-02 o Changed the negascout research so that instead of using -t.score as the upper bound it uses -alpha. This is an improvement since it does a better job of preserving the PV during the search. 11-SEP-02 o *Major* bug. I thought I was adjusting the mate values retrieved from the hash table, but it was incorrect. I know this was correct at some point in the past and I must have inadvertently screwed it up. Aw, the "joys" of chess programming!! o All mate/stalemate scores are stored in the hash table with an "infinite" draft, since these scores are *always* good and should not be overwritten. 20-JUN-02 o Changed the hash_t key in the main hash table to a union of ... union { ulong_t k2[2]; hash_t key; }; This allows the actual address to be pulled from the bottom 32-bits without moving having to perform the AND masking function on 32+ unused bits. o Fixed a bug in the run epd test suite driver. The program was printing out the expected solution move after finishing a problem. This is OK as long as the program is trying to find a matching best move. Unfortunately, this crashes if the program is trying to avoid a move since, in that case, there is no best move to print. o Modified the way an AVOID move is handled when running a test suite. Previously, the program counted teh position solved if it avoided the move three times in a row. Now it waits until the total time has elapsed and then compares the move to avoid with the move actually selected. This is a more accurate measure of the program avoiding the move. 16-JUN-02 o Fixed a bug in the "evaluate_[white | black]_6th_rank_pawns()" function that was keeping it from being called correctly. WAC2 gets solved correctly again. o Changed the amount pawns are extended in both the middlegame and endgame. Pawns are especially pushed in king/pawn-only endgames. 10-JUN-02 o Altered the tablebase code so that if it finds a draw it returns the value from the draw_score() routine instead of a simple DRAW score. o Removed the 2nd research since it wasn't really beneficial. o Tweaked the knight outpost code so that it likes blockading isolated- doubled pawns. o Tweaked the king safety code so that it doesn't like having the g-pawn (or b-pawn) if the king has castled queenside ripped away- especially if there are enemy pieces nearby. 08-JUN-02 o FIXED THE THREE-FOLD REPETITION PROBLEM- FOR REAL!!!! This was a major bug, from the beginning. The problem was that the repetition increment code was incrementing using "side" immediately *after* "side" had just made a move. This is WRONG!! The value of side stored in the repetition hash table is the side with the right to move in this position. This is NOT the side that just made a move. After I modified the code to store the correct value of side, the three- fold repetition code started working correctly- probably for the first time ever -- yeesh!!!??? o Removed the new repetition code from fixup_pv() since it was no longer needed. 06-JUN-02 o Fixed a *major* bug in the repetition hashtable check. It was only checking that the hash table "address" matched. It was not verifying that the hash key matched (oops?!). o I also added three-fold repetition checking to the "fixup_pv()" function. It now will return a draw score if the same position is repeated three times in the line it pulls from the hash table. 02-JUN-02 o Added a swapoff check for absolutely pinned pieces to determine if they can be captured favorably. o Added a UPDATE_HASHKEY() macro for debugging the dynamic updating of the hashkey. Compiling the program with the DEBUG_HASH switch set now actually debugs the hashing (in a very transparent way)!! 29-MAY-02 o Changed the repetition counter back to 2. Previously, it was 3 but the program was losing to many 1/2 points to 3-fold repetition draws. o Move the special move check after the captures and mundanes are sorted. 27-MAY-02 o Added a routine that handles the case when only a single pawn is left for either side in a king/pawn-only endgame. o Revamped the passed pawn advancement scores again. The advancement scores are now scaled by the total number of pieces left. o Enhanced the backwards pawns code since the program was getting into trouble because of too many backward pawns. 26-MAY-02 o Added a pinned piece penalty term in the evaluation function. o Increased the value of passed pawns. o Halved the bonus of the new single-piece trade-off term added earlier. 25-MAY-02 o Fixed the main bug in the "evaluate_outside_passed_pawns()" routine. o Updated the tradeoff routine so that the cases where one side has the only piece versus a bunch of enemy pawns is considered advantageous (which means that *hopefully* the program will steer for these type of endgames). o Killers are now cleared before a search. This was a bug that actually caused the engine to give away its queen (ouch?!). 22-MAY-02 o Some endgame evaluation tweaks. 16-MAY-02 o Tons of evaluation changes to the pawn code (and some minor rook on the 7th tweaks). This will need lots of tuning but it should play much better after I get it "right". o Determined that the "evaluate_outside_passed_pawns()" routine is broken. It can't be used until it is debugged. 13-MAY-02 o Changed the code that evaluates the pawn structures of the kingside and queenside castling positions (both current and future). The program will not access the castling bonus if the fianchetto pawn has been pushed and the fianchetto bishop cannot reach the fianchetto square in one move. This fixed the problem of the program pushing the g3 pawn to create a luft with the protective bishop missing. o Removed the repetition code that flagged two repeated positions as a draw *and* removed my "experiment" that was flagging a repetition of the position with either side to move as a draw because it casued the program to give away two rooks in a dead even endgame!!. 12-MAY-02 o Continued tweaking the pawn code. The king (both ours and the enemy) now moves toward the any passed pawns in the endgame. o Lowered the bonus for the rook behind a passed pawn. It was too large which was causing the program to lose games. o Modified the criteria used to determine when to call the functions "evaluate_[white|black]_6th_rank_pawns" so that these functions are called when we are in a knight/bishop vs. 2 sixth-rank connected pawns. o Added a sort routine for moving pawn promotions and pushes to the 6th and 7th rank to the head of the list when in the endgame. 10-MAY-02 o Completely rewrote the pawn evaluation code. This has the potential to be *much* better but it needs to be tweaked and tuned (of course). 05-MAY-02 o Rewrote the "rule-of-the-square" code. It should be more reliably *and* more accurate (heck, it might even be faster :) o Fixed some of the bad/good/active bishop code in the function "evaluate_black_bishops()". o Added a heuristic that penalizes a side if it has only one bishop and the pawns for that side are on the same color squares as the bishop. o Added an adjustable multiplier for the bishop mobility. o Implemented a pawn ram evaluator for king and pawn only endgames. o Added a routine to score the king opposition when in the endgame. 29-APR-02 o Add a heuristic to penalize a queen cut off from her king if there are more than X number of pawns. 27-APR-02 o Removed the "determine_cpu_speed()" function and instead simply set the the input search loop to 32,767 (0x7fff hex). If you run it on a slow machine you get what you get. o Completed the port to the Intel 6.0 C/C++ compiler (very nice increase in speed using Intel's compiler). The Intel compiler basically compiles it out of the box- including the inline assembly and the "long long" data types!! o Emulated the Unix gettimeofday() function under Windoze. This version of the program now compiles under Gnu's gcc, Intel's icc 6.0 compiler and Microsoft C/C++ 6.0 (sp5). o Fixed a bug in the three-fold repetition code. The program was checking the *wrong* side when querying the repetition hash table, causing it to miss saving three-fold repetitions when down in material. o More "C-ification" tweaks to speed things up. 23-APR-02 o Finally added a successful winning/even/losing captures pass to the negascout search. o Started "C-ifying" the C++ code by converting the "const int" variables into C-style #defines. o Added a bitmap and counter to track the number of pawns for both sides. 20-APR-02 o Experimented but ultimately removed the double nullmove heuristic to catch zugzwang positions. I wasn't seeing any real benefit. o Added code to catch the missing tablebase problem. This can occur when we transition from a position that is covered by a tablebase to one that is not due to either a capture or promotion. If we don't check for this we could get back a losing move from the search. This will probably get refined in the future. o Reordered the promotion and promotions with capture so that Queen promotions are generated first. Occasionally, when the program was about to promote a pawn to a piece that was going to be immediately captured it would promote to a rook, bishop or knight instead of a queen. This wasn't really incorrect just a little unorthodox. o Tweaked the castling bonus code and the king safety. 16-APR-02 o Fixed a subtle tablebase bug that cost the program half a point on ICS when it threw away a lone king vs. lone king draw. The program was in a KR vs. KR endgame when the opponent took the engine's rook. The program could have easily captured the rook for a trivial draw, but instead it moved its king away!! The reason- because the EGTB probe code did not realize that K vs. K is a draw- ouch!? Basically, this is was a case of needing the lowly 2-man tablebase KK. The probe code now realizes that this is a draw and behaves appropriately. 15-APR-02 o Added the "egtb-cache" command which allows the user to dynamically resize the cache used by Nalimov's code when it probes the various endgame tablebases. o Increased the size of the evaluation hash table to 1 MEG. 14-APR-02 o Fixed a number of EGTB bugs that showed up when evaluating draw positions. I believe the massive slow-down in search speed is caused by the program accessing the hard-drive when searching the larger tablebases. 13-APR-02 o Fixed a pondering bug that cost the program about 75 rating points on ICS. The program was pondering fine except when it found a mate. Then it went loopy (that's the technical term ;). This was *exactly* the same problem it had when in analysis mode, and the fix was the same- don't exit the search early when a mate is found. o Added a "contempt" factor so that the program will try to avoid draws with lower rated opponents on ICS. 11-APR-02 o Experimental speed tweaks, nothing major. o Fixed a bug in the pondering code. The start time for the program was not being reset when the opponent made the move predicted by the program. o Altered the pondering strategy so that the program does not ponder if the current position is in the opening book. o Added a second (fixed-size) replace-always hash table. 03-APR-02 o Revamped the opposite castling logic so that the program is more likely to open files around the king. 31-MAR-02 o Increased the tradeoff bonus by approximately 2x. o Reduced the number of nodes searched before the engine checks for user input. Hopefully, this will make the program less sluggish in analysis mode. o Added a check for KPPP vs. K endgames since this is usually (99% of the time) an overwhelming advantage for the side with the pawns. 30-MAR-02 o Added a penalty if the program is in a rook vs. rook endgame and it has at least one extra pawn. The idea is that single rook vs. single rook endgames are drawish. It would be easier for the program to win by trading rooks. o Tweaked the king safety so that it keeps a few pawns around :) o Added a check to determine if the program is castling into an unsafe area of the board when the opposite side is safe (not a smart thing). 17-MAR-02 o Added the two bishops bonus to the opening evalution function. This was added so that the program will be less likely to give away the two bishops advantage in the opening. o Give a bonus for having a single bishop versus a single knight in the endgame when the center is open and there are pawns on both sides of the board. o Increased the penalty for doubled pawns and decreased the penalty for backward pawns on an open file. o Increased the bonus for a pawn in the center (in the opening) to some ridiculous value to *make* the program to get it's pawns out into the center. I more than likely change this after I see how it plays. 13-MAR-02 o Removed the principal variation stuffing at the root since it was making the program significantly weaker tactically. o Fixed a serious bug in the SKIP_NULL and SKIP_NULL_EXTEND code. Essentially, it wasn't working at all!! The problem was that the SKIP_NULL* flag was being stored in the "bound" variable, which sounded good except that every move stored into the hash table effectively overwrote the SKIP_NULL* information- aarrrggghhh!! I fixed it by adding a new field to the hash slot (I replaced the "age" field since it wasn't being used). o Altered the draw by three-fold repetition count so that 2 hits during the search gets scored as a draw. Of course, at the root it's handled normally. Hopefully, this will reduce the number of three- fold repetition draws when the program has a better position. 01-MAR-02 o Reinstituted the Euclidean distance tables. The tables based on the king traversing an empty board were kept, but renamed to "KingWalk" and "InvertedKingWalk". The old distance tables were better for judging king threats but the new "walking" tables will still be useful in evaluating certain pawn endgames. o Revamped the rook evaluation code that determines if the enemy king is trapped on the back rank when a rook is on the 7th. This is now a 2nd- order evaluation, slow but accurate. o If valid, the first move of the last ply's principle variation is searched first at the root. 24-FEB-02 o Changed the way the search is handled in analysis mode. Previously, whenever a mate was found the search would be truncated and then restarted. Now the engine emulates crafty and yace in that it continues searching (this seems like the more natural way to handle it). o Added a CPU measurement loop in the engine initialization code. The mask used to determine how often the program checks for user input is computed dynamically on the fly. This should help avoid penalizing fast machines by making them check for user input more often than necessary. 23-FEB-02 o Fixed a bug in the qsearch that in rare circumstances was causing the program to skip captures based on the results from the swapoff routine or even the futility cut-off logic. Very bad, since this would occasionally cause the program to erroneously think it was mating or being mated. o Implemented the '.' command in analysis mode under xboard. o Remove the time() commands calls and replaced them with a custom routine "read_clock_in_centiseconds()" (which uses the gettimeofday() function). The time for a move is stored internally in centiseconds instead of seconds. 18-FEB-02 o Added lazy evaluation to the evaluation. This needs to be watched closely to ensure it doesn't screw things up. o First pass at implementing tree search logging in the search routines. This needs to be worked out for the Windows version. o Fixed a bug where the captures and the mundane moves were being sorted AFTER the check for duplicate moves against the special "cutoff_list". The result was that the entire purpose of the cutoff list was being defeated and in some cases the same move was getting searched twice. 05-FEB-02 o Loosened the requirements for a test passing when running a test suite. The problem was that a move would fail-high and then on the research fail-low, so valid moves that matched the solution move were being thrown out. 02-FEB-02 o Added the "toolbox::basename()" function. This routine extracts a file's basename from the fullpath name provided. o Converted the distance arrays "Distance" and "InvertedDistance" over from Euclidean distance to minimal moves from one square to another distance. This makes more sense for chess, since it is definitely a non-euclidean world!! o Added an additional "passed_pawn()" function to the board class. o The program now asymmetrically penalizes pawn rams if it is ahead in material at the root. This is a first attempt at trying to get the program to open the position if it has an advantage or close the position if behind in material. o Added an asymmetric penalty for opposite color bishops in the endgame. 31-JAN-02 o Added a hash matching function for the qsearch that doesn't check the draft. o Also removed the code (which wasn't being used) that set the draft in the hash match function. 27-JAN-02 o Fixed a couple of minor bugs in the EGTB code (OK, draws were being ignored :) but besides that they were minor). o Added EGTB checking to the root. If the score at the root is a draw then the program does one of two things (1) It culls losing moves or (2) if we are in swindle mode the engine selects the draw move that prolongs the game the longest, (in hopes that the draw will turn into a win by letting the opponent make a mistake). o The "myname" command to xboard passes the command-line name the program was invoked under. This solves the problem of two versions of the engine running under xboard with the same name. 25-JAN-02 (experimental tweaks) o Changed QSEARCH_SWAPOFF_DEPTH to 6. This defines how deep the qsearch goes before using the swapoff function to resolve the capture search. o All non-captures in the qsearch are sorted with the history table. o The null-search is now searched with a window of <-beta, -beta+1>. The value of alpha is no longer modified using the value returned by the null search. o Added a routine to check and supplement the PV at the root after each search iteration returns. This has had a dramatic (and good) effect on the veracity and length of the principal variations returned to the user. 23-JAN-02 (ICC rating 2150 at 15 3) o Changed all the instances of "-CHESS_INFINITY + MAX_DEPTH" to "-CHESS_INFINITY + 1000" to cover all the tablebase mates. o Added pawn pushes to the 6th, 7th and 8th ranks in the qsearch. o Added queen, rook, bishop and knight normal checking moves into the qsearch. Adding this actually *SHRUNK* the search tree by 25% for the WAC test suite!! All of this needs to be tuned, so more than likely I'll be running a number of experiments to get this all working together in a somewhat more optimal fashion. Still it's very encouraging. 22-JAN-02 (experimental tweaks) o The program now extends single-pawn forks of knights and rooks. o The history table is now scored with "MAX_DEPTH-depth" instead of "MaxDepth-depth". o Cleaned up a bug in the "clear_killers()" routine and simplified the killers replacement code. o Limited the check extension. If move just made placed the enemy king in check we extend the move a half ply. If the program is not in check itself we extend the move an additional half ply. 21-JAN-02 o Fixed an king safety bug that highlighted an interesting characteristic of the evaluation function. The program was being penalized because its kingside pawn cover was ripped up (so far, so good). To get rid of the penalty it intentionally moved its king in order to lose *all* castling rights!! The lesson to be learned is that to encourage certain behavior it is better to give a bonus instead of a penalty. In fact, it could be argued that penalties should never be used!! Probably, extreme, but a *very* interesting line of thought. o Increased the bonus for castling earlier in a variation (the "MAX_DEPTH - depth" term basically). 20-JAN-02 o Added the ability to read the condensed single-line EPD file format. o Added the ability to run an EPD test suite using the "avoid" move option. The idea is that the position is counted corrected if the program avoids the move given for "N" consecutive searches. o Modified the program so *all* the echo lines entered are saved and displayed when that particular program is run. o Added an additional ID field to the test suite structure. o Tons of cleaning in the EPD driver code (I'm sure I introduced new bugs, it's definitely time to run BoundsChecker on this mess). 19-JAN-02 o Increase the penalty for making skewed piece for pawn trades. The idea is to discourage the program from trading a piece (or pieces) for the numerical equivalent number of pawns. Ultimately, this usually turns out poorly in the endgame. o Changed the values of the pieces to discourage even more the bad trades mentioned above. o Added repetition checking to the qsearch. o The program now checks if one side has insufficient material to win. If true then returning a positive value for that side is incorrect. For example, WHITE King + Knight vs. BLACK King + pawn is not a "good" position for white. White should be striving to avoid a loss. We accomplish that magic by subtracting the *losing* sides pieces material away from the score, thus the best the program can do is draw. 18-JAN-02 o Fixed an bug in the eval function picked up by Microsoft C++. Again proving you should compile your code on at least two separate compilers, (or run lint on it- whichever comes first). o Update the == and != operator for the board_t class to reflect all the new fields that have recently been added to it. o Made "CurrentLine[]" a global variable so it can be accessed anywhere, (which is useful for debug purposes). o Updated the internal board class debug routine. o Fixed a *major* bug in the qsearch usage of the hash move. Basically, the hash move was being checked for correctness before being used (as it needed to be) but the move list counter was being incremented regardless of the outcome of the check?! A big no-no since it should only have been incremented if the move is actually valid. 14-JAN-02 o Fixed a bug in the opening book probe code that would cause the engine to occasionally play an illegal move (basically, it would play a legal move for the other side!!) 13-JAN-02 o Reintroduced the opposite castling detection algorithm after a nasty loss on ICC. This will need to be tweaked over the next few days. o Increased the penalty for placing a bishop/knight on d6 (or d3 for white) if a pawn is on d7 (d2). Again, necessary because of a loss on ICC which was directly related to this problem. o First implementation of an "orcale" root evaluation routine. o Fixed a bug where the piece table values for the pawns were not being added into the evaluation function. o Added a penalty for bringing out the queen early. o Started to fully utilize the major/minor and population counters in the evaluation function. o Tons of evaluation tweaks. 11-JAN-02 o Tweaked the time algorithm code (plus corrected a bug that caused the program to lose a game on time on ICC). o Added "major_piece" and "minor_piece" bitmaps to the engine. Minor pieces are knights and bishops. Major pieces are *all* queens, rooks, bishops and knights (this is different from the classic definition but works better for what the engine needs). 10-JAN-02 o Fixed the tablebase problem, gcc 3.0.2 was screwing it up. Switching back to gcc 2.95.2 corrected the problem. 09-JAN-02 o Added Nalimov endgame tablebase support!! I used the routines off Hyatt's ftp site and it went *mostly* OK. There were definitely a few false missteps along the way but most of the kinks have been worked out. o Created a new "movepv_t" structure that incorporates a normal move and a flag field. This flag field marks the principal variation move as either a normal move returned by the search, a move retrieved from the hash table, the *last* move returned from the hash table, the first move found in the EGTB, or an optimal move returned by the EGTB. o Added a "whisper()" and "kibitz()" function for broadcasting on ICC. o Fixed a bug in the "analyzing_findmove()" function. After the search exiting it was not setting the "engine_t::is_pondering()" flag to false. This was causing xboard to crash whenever anyone tried to load a new position while the engine was actively analyzing. 05-JAN-02 o First pass at "Internal Iterative Deepening". It seems to be working well (wac 71 showed a remarkable improvement). 04-JAN-02 o Added in the extenstion reduction by 2 when the depth is greater than 2 x the maximum search depth for this iteration. It seems to work fine with no loss in tactical ability but a decent reduction in tree size. o Modified the search so that it is multi-pass (i.e. it performs local researches of interesting branches). On the first pass through the program only extends checks and null-threats. Pawn pushes and recaptures are only tried during local researches. As counter- intuitive as this sounds it works because the first pass loads the hash table with promising moves which causes the researches to be lightening fast. The overall effect is a *reduction* in the final tree size. 03-JAN-02 o Substantial bishop evaluation tweaks. I also fixed a bug in the trapped bishop code. Removed the bishop mobility term. Added a fianchetto penalty and crude bad bishop code (we'll see how it works out in the near future). o Also altered the value of the piece tables based on the article "Learning Piece-square-Values using Temporal Differences" from the Dec. 1999 ICCA Journal (Vol. 22, No. 4). 02-JAN-02 o Tweaked the tradeoff routine to equate knights and bishops. o Tweaked/cleaned up the pondering code (this is still the most problematic part of the program- and the most buggy!!) o Major revamp of the time algorithms. Specifically, the incremental time controls supported on ICC were completely broken. To determine the time for each move under the new time control the algorithm is: if {time left > 10 * INC} time = time left / 40 moves if (time left <= 5 * INC} time = INC / 2 else time = INC o This version (under the name KnightMere 0.1) played its first three games on ICC. And to top it off it won all three. 31-DEC-01 o Added a trade off bonus when the program is ahead by at least two pawns. It will try to trade down into a (hopefully) won endgame. o Increased the values of passed pawns since the program was not giving them enough attention. 28-DEC-01 o Created a number of wrapper functions around some of the fast inline evaluation functions. 27-DEC-01 o Fixed a memory overflow bug in the PV display routine. o Corrected a bug in the analysis/ponder state-machine code (who knows maybe it actually is correct now ?!?!-- I wouldn't bet on it though =:o o Created and implemented the ADD_SCORE and ADD_POS debug evaluation macros for use in tuning and logging the evaluation function's scores. o Started major restructuring of the evaluation routines. o Reversed the order that PCAPTURES and PROMOTION moves are generated. This has the advantage that now if two promotion moves (such as pawn->bishop and pawn->queen) both give mate the program will pick the pawn to queen promotion. This will help on some test suites, other than that it's simply an aesthetic thing :) o Added some additional initialization messages when the program starts (basically, to help out when using BoundsChecker, which takes forever to start and is very good at playing dead). 25-DEC-01 o Sped up the routine "move_t::valid()". o Significant tweaks to the rook evaluation routine. 19-OCT-01 o Tweaked the king evaluation, outside passed pawn code and rook scoring. 13-OCT-01 o Tweaked the "load_command_fifo" routine again. o Split the "sort_captures" function into two functions, "score_captures", and "score_captures". 11-OCT-01 o Fixed a bug in the "load_command_fifo" routine that was cancantenating two commands together when the input into the buffer was *exactly* 80 characters (the size of the buffer) and the newline character after the last command fell in the 81st location (i.e. it was the first character of the next read operation). o Modified the "exit_the_search" routine during analysis/ponder mode so that it checks for any pending FIFO commands as well as user key strokes during the search. 08-OCT-01 o Finally, fixed pondering and a minor analysis mode bug. This is still by far the buggest code in the engine and should eventually be rewritten. I also had to add a label in the io input routine (this is spaghetti code at its best). 07-OCT-01 o We have started bombing Afghanistan!! Time to glow Mr. bin Laden!! o Rewrote the outside passed pawn code to both speed it up and make it more accurate. o Added an initialization routine for the main hash table. It initializes the new hash values array "RandomFromTo[ 12 ][ 4096 ]". This array combines the from/to hash keys into a single hash key. The elimination of the second hash key value resulted in a significant overall speed up. 30-SEP-01 o Modified the qsearch so that it discriminates when in-check. Previously, we always generated all check-replys when in check, but that is not necessary unless the side in check was in check every time it was that side's turn to move in the qsearch. Sounds confusing, huh. The best explanation is the code. A decent amount of comments were added to the quiescence search. 29-SEP-01 o See the Sep. 29th changes in the EVAL file. 26-SEP-01 o More evaluation tweaks (mostly king safety). This will probably go on for awhile as I revamp the evaluation routine. o Added the max search depth and average search depth information to the HTML output data. 24-SEP-01 o Additional changes and tweaks to the king safety code. 23-SEP-01 o Start of a major rewrite and initial vamp of the evaluation function. 18-SEP-01 o Finally, a recapture extension that seems to work!! o Fixed a bug in the new variation routine. 16-SEP-01 o Tweaked the king safety evaluation. Specifically, made the program more reluctant to open files around the king. I also adjusted the piece table values to be more neutral overall. 15-SEP-01 o Rewrote the "Board.variation()" routine so that it displays the current move number when it puts together a variation to show. o Added a "castling_penalty[]" array to replace the retraction of the castling penalty when making and unmaking a move. 14-SEP-01 o Replaced all the instances of "!side" with define "CHANGE(side)" which underneath is the XOR ((side) ^ 1) optimization. 11-SEP-01 o A DAY THAT WILL LIVE IN INFAMY!! THE BASTARDS RAMMED THE WORLD TRADE CENTER!! o Changed the cut-off for when the swapoff function is used in the quiescence search to "depth+8". o The null-search is once again used in the endgame (except for pure king and pawn endgames) with the change that it is the search is is conducted one ply deeper. o This version solves 297/300 WAC problems at three minutes a move. At 30 seconds per move it solves 291/300. At 5 seconds/move it solves 278/300. 09-SEP-01 o Added the "phash " command to dynamically resize the pawn hash table when the program is running. o Reduced the number of hash probes to two instead of four. Profiling was showing that this function was taking 8-9% of the program's total time (too much). o Added a search_root() function that will eventually develop into the "Orcale" type routine for sorting moves at the root, loading the piece square values, setting the piece values ect. 08-SEP-01 o Added the "hash " command to dynamically resize the main hash table when the program is running. o Changed the values of the bishops, knights and queens so that the program is less likely to trade a three pawns for a minor piece. o Constrained the quiescence search so that it uses swapoff() after going four ply past the normal search and the quiescence search boundary. 07-SEP-01 o Changed the "protover" routine so that it accepts xboard protocol versions greater than 2, without crashing. o Added the "Leaf???" variables to the engine. Using these variables it is possible to determine the maximum extension depth reached by the search. It is also possible to determine the average depth of the reached by the engine. 03-SEP-01 o EVAL 30-SEP-01 changes 30-AUG-01 o Added a major revamp to the user input code to fix the program's interaction with xboard. Xboard was sending commands while the engine was searching that were being ignored. To solve this situation I added a command FIFO deque which is stores the commands until the search completes when the are then processed. 29-AUG-01 o Changed the program so that it now resigns when far enough behind. Related to this is that the program now accepts a "resign" command that sets the resignation threshold and the number of consecutive moves that must exceed the threshold before the program resigns. o Added the "load" command as an alias for the "loadfen" command. o Fixed a bug in the testmode code. The selected move for a ply is NEVER the move returned by a fail-low search. In the case of a fail-low search we keep the last known good move from the previous N-1 search. Unfortunately, the exit early test code was comparing this last good move twice whenever we failed low, giving bogus good solutions. 26-AUG-01 o Added the xboard command "computer" as a no-op so the program would quit complaining about not recognizing the command (it was harmless, but annoying). o Added the "depth+1 > Maxdepth" check back into the null search. Leaving it out was causing the program to play weaker tactically. o Added a check to the null-search criteria so that if the program is in kings and pawns only endgame it skips the null-move. This helps avoid problems with the many zugzwang positions that seem to arise in these endgames. 25-AUG-01 o Finally, added the code for the program to mate with a king, knight and bishop versus a lone king. o Altered the criteria for performing a null-search. Previously, the null-search was not performed at all in the endgame. This was too restrictive. o Pulled the "depth+1 > Maxdepth" check out of the null-search criteria. This should have come out earlier but it was overlooked. o Added a "engine_t::globalNull" variable to turn off null-searching completely during a search. This was necessary to implement the mop-up routines reliably. 24-AUG-01 o Finally, added a check during the search for the "infamous" fifty-move draw rule. The program recognizes and claims draws under this odd-man out rule at both the root and during the search. o Cleaned up the "game_over_msg" routine. It has a much more coherent structure now. o Eliminated the "queen_[rook | bishop][SIDES]" arrays. Instead the program uses w_queen_rook, b_queen_rook, w_queen_bishop, etc. o Converted the "occupied[SIDES] bitmaps into w_occupied and b_occupied. o Major evaluation tweaks, especially to the knight scoring. Early, indications are that the improvements are positive- time will tell. o Corrected a bug in the mopup evalutation routine that was causing the program to lose in simple mop-up situations. 21-AUG-01 o Revamped the way the null-search was invoked. If we are past the boundary between the normal search and the quiescence search then the null-search uses the qsearch instead of the normal search. I also lifted the restriction that the null-search could only be invoked during the normal search. All-in-all it seems to have sped the search up considerably without any loss of tactical strength. o Added some additional inline assembly to the bitmap.h file to clear the least-significant bit of a bitmap. o Added the 1st pass at the "outside passed-pawn decoy" code. 20-AUG-01 o The qsearch now passes the "in_check" information as part of its parameter list. o The check extension in the "extend()" function is set to "extension -= ONE_PLY" o The "qsearch" command was removed from the user command list since it was basically worthless. 19-AUG-01 o Merged in the old negascout search routine with the new updates. o Added futility-pruning into the quiescence search. It reduced the size of the game tree without any appreciable tactically loss in strength. o Removed the "eval" command from the user command list, since it was basically worthless. 15-AUG-01 o Revamped the "check_reply()" functions to be faster. o Added the inline "piece_on_square()" function to replace all the inline (error-prone) "board8x8[] & 7" statements. o Broke out the "attack_on(side, square)" function into two additional functions "attack_on_[white|black](square)" and inlined them. 13-AUG-01 o Finally, I believe I've located and fixed the principal variation truncation problem. Principal variation moves (which are considered EXACT moves when stored in the hash table) were always being stored with a maximum draft value. This is OK if the move is mate or stalemate but is incorrect for normal principal variation moves. By giving all PV nodes equal weight we incorrectly generated too many matches. When these bogus hash hits occurred there were no principal variation nodes below the node that triggered the invalid match and the main line PV was incorrectly truncated. o Fixed a bug in analysis mode that was causing the program to jump in and out of analysis incorrectly when a mate or stalemate was found. o Altered the inline assembly code in "bitmap.h" so that it uses an registers available instead of just the eax and ebx registers. This should allow the compiler move latitude when compiling the code, (which will hopefully result in faster code). 12-AUG-01 o Altered the "read_player_input()" function to return an STL std::pair instead of an integer only. The boolean value tells the engine wheter or not it should continue to search while the integer value is used to set the state variable for the program's main FSM. This change resulted in a much cleaner structure and I was able to make the state variable local to the FSM in main, reducing the module coupling between the FSM and the input routine. o Castling rights for the side with the right to move are now stored in the evaluation hash table. This allows evaluation terms dependent on the castling rights to be used anywhere in the various eval routines. 11-AUG-01 o Finally, tracked down the problem with the evaluation hash table. Previously, the score returned by the evaluation hash table was not matching the actual calculated score. The problem was two-fold. First, the hash key for the position was not being updated in the quiescence search (oops,... :( And secondly, depth dependent information was being calculated and stored in the hash table. Specifically, the castling rights for the current depth were being stored causing mismatches. o Tracked down an additional hashing problem with the way the pawn scores were being stored and retrieved from the table. The values being stored in the pawn hash table were the raw values returned by the "evaluate_pawns()" routine (so far so good). Unfortunately, when the score needed to be calculated on-the-fly it was being multiplied by a scalar, thus the two values were no longer matching. 10-AUG-01 o Started adding debug code to track down some odd hashing anomalies. 09-AUG-01 o Tons of modifications and tweaks- o The quiescence search as well as the normal negascout search is called when we try a null-move. o Modified the fast capture generation routines so that they are even faster :) o Added killers back into the mix. 21-JUL-01 o Tweaked the evaluation code and the search. 20-JUL-01 o Continuing to clean up the various modes and the interface in general. o Removed the global variables "Analyze" and "Ponder" out of main.cpp and instead moved them into the engine_t class. The analysis and ponder states are now set via methods of the engine class, which is cleaner. o Implemented the xboard "setboard" command. 18-JUL-01 o Continuing to tweak the new single-thread interface. 17-JUL-01 o Ripped out the curses interface and all the multi-threaded code. It was too hard to debug. At least now I have a shot at getting it to run bug free. 14-JUL-01 o Modified the "grab-a-snapshot" routine so that it uses the changes logged in this file to generate the build number. o Performed a general cleanup of the files in the "console" directory. 24-JUN-01 o Divided the evaluation function into three modules. One module contains common routines, while the other two contain white and black routines. o Corrected a bug in the pinned pieces routine. Previously, the routine was scoring pawns pinned to the king by a rook or queen along a file as "absolutely" pinned (which may or not be true). o Added additional debugging routines for the evaluation routines. 03-FEB-01 o Modified the data structure used to define the move types. The new data structures no longer need to be shifted to extract the move type, (which is a huge speed up). 06-JAN-01 o Added multithreading under both Windows and Linux. o Added support for the xboard "analyze" command. The cool thing about this is that it will run under ChessMaster 8000!! 31-DEC-00 o Started implementing the 2nd Xboard protocol supported in Xboard 4.2.x. 30-SEP-00 o Fixed a major bug in the time allocation algorithm. The program currently uses a simple-minded approach when a search fails low. The move returned by a fail-low search is not reliable, so the program take a two-pronged approach to the problem... (1) It doubles the basic search time. (2) It only saves a move if the search was a "good" search. Previously, the program was throwing away won or even positions by making the fail-low move. 26-SEP-00 o Fixed a bug in the qsearch. Essentially, what was happening is that when the program was in check (in the the qsearch) it was dropping the hash move because the history sort was moving it out of the first slot. 24-SEP-00 o Modified the program (and the Makefile) so that the program can be compliled for both single-threaded and multi-threaded operation. When configured as a single-threaded application pondering is disabled. o The program can now be compiled under Visual C++ 6.0 as a single- threaded application. o Changed the overloaded int operator for bitmap_t so that it can be used on "const"ant bitmap_t objects. o Ran BoundsChecker and TrueTime on the program and actually discovered a number of bugs, that were potentially serious. They ranged from memory overwrites to use of uninitialized variables. Once again, BoundsChecker proves it was worth the money I spent for it. o Added a boolean variable "best_move_valid" to the qsearch. o Added a new INVALID_BOUND constant to the hash constants (EXACT, LOWER, UPPER, SKIP_NULL, etc). o Overloaded the "strtok_r()" function so that it compiles under Visual C++ 6.0. o Set it up so that the directory functions for Windows are also supported. This was necessary because Windows doesn't support the DIR structure that LINUX (and UNIX) use. 20-SEP-00 o Fixed a bug in the search. Essentially, what was happening was that for very deep searches the array bounds of some of the MAX_DEPTH arrays were being overwritten. The fix was to simply increase all the arrays of MAX_DEPTH to MAX_DEPTH+1. This only showed up because the new athlon searches so much faster than the 300-Mhz Pentium II. 13-SEP-00 o Added evaluation code for knight outposts. 10-SEP-00 o Revamped the eval functions. Started the conversion to a local global structure for most of the functions that access the chess board. 30-JUL-00 o Added the "printL" function. This function only writes to the log file. 07-MAY-00 o Modified the timed search so that if we fail-low on the final iteration we double the search time in the hope that we can resolve the failure. o The history tables are no longer cleared between searches. Instead the values are scaled back. This lets us take advantage of the old moves stored in the table, without compromising the new search. o Added a "weak back rank" term to the middlegame and endgame king safety routines. o Added the recapture extension. I'm still not sure this helps but I'm going to leave it in for awhile and see how it performs. 05-MAY-00 o Added the "DEBUG_EVAL" tick define to the evaluation functions. Essentially, it prints the values of each evaluation term. o Reworked the positional components of the eval routines. o Added hashing to the quiescence search. o FAIL-LOWs (bound set to UPPER) no longer have a move stored in the hash table, since they have no move associated with them. 01-MAY-00 o Changed the extend function so that the "one-move-only" and the "null-move" threat extension increment the ply level by 3/4 of a ply instead of 1/2 ply. o Altered the fourth slot in the "add" function for the hash table so that it is a "REPLACE-ALWAYS" slot. 30-APR-00 o Changed the type of values that are stored in the "board8x8[64]" array. Previously, piece values were stored in this array. Now W_KING, W_QUEEN, ... B_KING, B_QUEEN, etc. array indexes are stored there. To obtain a piece value from this array you need to index into the "PieceValue[]" array. W_* (the white) indexes and B_* (the black) indexes are separated by 8. The advantage of this is that KING, QUEEN, etc. base indices can be obtained by ANDing the "board8x8[]" value with 7. Which is a useful property in a number of places in the program, especially now that the KNIGHT and BISHOP have the same value. o Created a new "generate_fast_captures" that generates captures at a high rate of speed and then sorts them using the "swapoff()" Static- Exchange Evaluator. It seems to be as fast as the old routine which only sorted things in LVV/MVA order (which seems to be inferior to SEE). o Tweaked the king scoring in the endgame, so that it wants to be more involved in the action. o Added a number of new "ClearMask[]" arrays analogous to the "SetMask[]" arrays added a while back. This includes the masks for clearing bits in the rotated occupied maps as well as the normal map. o Added the age field back into the hashtable. o Added additional criteria for entering into the endgame. If both queens are gone and both sides have one or no rooks then let's call it an endgame and get the king involved. o The random number generator is now seed with the time in the engine_t constructor. o Removed the trade-off bonus out of the evaluation function since it was causing large abberrant positional scores to be generated. o Changed the "largest_positional_score" to an array. One value is now keep for both sides. o Added a lazy evaluation tick define for the eval function. 29-APR-00 o The program now scores (at a very primitive level) bishops and knights trapped in the corner. This is a first pass at an "entombed" piece evaluator. o Changed the value of a knight and bishop so that they are now equal. 26-APR-00 o Modified the engine so that it prints out the solution depth of each test position solved correctly in the lab. o Removed futility-pruning after extensive testing showed that it didn't really give any advantage. It needs to be revisited in the future. 25-APR-00 o Added a PASS/FAIL indicator to the log file for each test position when running a test suite. Useful for post processing of the log file. I also added a START/STOP indicator for each test position. o Modified the "tally-tests.pl" perl script to take advantage of the PASS/FAIL indicator and the START/STOP info. o Added a tick define EVALUATE_MATERIAL_ONLY in the evaluation routine. 23-APR-00 o Altered the hash table so that it tries four slots instead of only two. o Changed the criteria for adding new entries to the hash table. No EXACT moves are ever replaced, since they only take up a small portion of the hash table and are useful for restoring the PV when probing the table. o Increased the size of the evaluation hash table to 1-MEG. I also changed the evaluation hash table so that it utilizes four slots instead of two. o Stalemate scores are no longer saved to the hash table. o Broke the SPECIAL_MOVES_PASS into HASHMOVE_PASS and PV_PASS. The principal variation move is now tried after the hash move, captures, and in-check replies. 22-APR-00 o Added futility forward pruning to the search. The idea is fairly staightforward, if the current position seems uninteresting (i.e. no extensions have been triggered) and the side moving is behind in material plus the move being made is NOT a capture then we enter the quiescence search one move early. One thing to note, the evaluation of the material must be the material score at the root of the tree. This avoids the problem of being behind in material (i.e. if one side is basically losing the game) which would inappropriately force us into quiescent search all the time. 02-APR-00 o Discovered (and corrected) a fairly major bug in the "generate_captures" routine. The moves were NOT being generated in MVV/LVA order. They were being generated for each piece correctly but were being generated in an incorrect fashion across a specific piecetype group. For instance, if a knight resided on e4 and another knight resided on f6 then the moves generated for each individual knight were correct by when the moves for each knight were added together they would be out of order. This is fixed now with a simple bubble sort. o Removed the "DRAFT(depth)" statements out of the evaluation function. o Added code to advance the pawns when the kings are castled on opposite sides of the board. 30-MAR-00 o Inlined the x-ray attack support function for the "swapoff" function. o Changed the "DRAFT(depth) / 4" tempo code in the evaluation function to DRAFT(depth). o Cleaned up (and rearranged) some of the code in the evaluation routines. 27-MAR-00 o Continued to tweak the king safety code. I added the "breezy-protection" heuristic to the king safety code for both the opening and middlegame. This code attempts to evaluate the exposure level of the king and penalize exposing the king by removing his pawn cover for no good reason. 25-MAR-00 o Added code for evaluating the safety of a king if the pawns in front of the king are fianchettoed. o Added code discouraging the moving of the bishop pawn on the side the king has castled on. o Scaled back some of the positional scores because some of the king safety elements were starting to overwhelm everything else. 20-MAR-00 o Minor modifications to the search function. The function "update_pv" is now called only after verifying that the hash table did not return a mate score. o Added additional pawn scoring debugging code was added to the evaluation function. 17-MAR-00 o Modified the "epd-enumerate" command so that the number of positions being enumerated is given as well as the commands progress while running. o Modified the functions "king_in_check()" and "attack_on()" so that the (expensive) functions rook_attk() and bishop_attk() are only called if we actually have a queen, rook or bishop on the board. o Sped up the move generation routines to take advantage of the "queen_bishop/rook" bitmaps. o Tweaked a number of evaluation parameters for the engine. Specifically, the king safety parameters and the piece on d3/e3 or d6/e6 blocking a king/queen pawn. o Altered the callback function to display more of the parameters the engine uses to evaluate a static position. 16-MAR-00 o Reworked the pondering logic. This is still (by far) the shakiest code in the progra