Iterate then iterate on your mistakes
I love this title as is it really defines the way I am working on my little search engine. I am making a series of decision that amount to getting each iteration completed as fast as possible. This means making a series of decisions that are at best maximally simple and at worst dumb, based on the idea I will fix them in a later iteration.
When faced with a problem the first question I ask is "Is there a dumb solution that lets me progress?". If there is a dumb solution I use it with the expectation I will revisit it in the future. Yes, I have already started to revisit some of these decisions and usually it is because the performance was not great once the data got bigger.
Making things fast is an enjoyable process so if revisiting means making things fast then I am probably going to enjoy this approach to coding.
One feature of this simple as can be approach is the code can get a bit messy. So each iteration I make sure it includes a task to clean up the code in some way. Just something simple like break apart a class or clarify some function names, there is not much code at the moment and this fix the mistakes of the last iteration seems to be keeping things clean enough to progress but this code is not going to be entering any shows
Each iteration is only an evening or two sat at my desk and consists of about 4 or 5 tasks. Meaning about 20% of the tasks generated deal with keeping things clean.
Now what features have I been adding? Well the search engine now has the ability to search for multiple words, the sqlite database now has index to speed things up be a factor of about 100 and the web crawler is a little bit more selective on what links it follows. Of course the results are still terrible as I have not implement any form of rating of pages yet.
Oh and I created a very simple website that lets me perform a simple search. This is another example of as simple as possible as its output does not even let you click the results and be taken there, that is for the next iteration.
I had a quick look around for python web frameworks as there was not point having the cognitive load of another language without a good reason. Bottlepy looked the simplest to get going as it is a simple file with no dependencies and then I created about 40 line python file including the in-lined HTML for page creation. And I was done. Further refinements will come in later iterations.
It is not publicly available as this is definitely still a for fun not showing type of project.
When faced with a problem the first question I ask is "Is there a dumb solution that lets me progress?". If there is a dumb solution I use it with the expectation I will revisit it in the future. Yes, I have already started to revisit some of these decisions and usually it is because the performance was not great once the data got bigger.
Making things fast is an enjoyable process so if revisiting means making things fast then I am probably going to enjoy this approach to coding.
One feature of this simple as can be approach is the code can get a bit messy. So each iteration I make sure it includes a task to clean up the code in some way. Just something simple like break apart a class or clarify some function names, there is not much code at the moment and this fix the mistakes of the last iteration seems to be keeping things clean enough to progress but this code is not going to be entering any shows
Each iteration is only an evening or two sat at my desk and consists of about 4 or 5 tasks. Meaning about 20% of the tasks generated deal with keeping things clean.
Now what features have I been adding? Well the search engine now has the ability to search for multiple words, the sqlite database now has index to speed things up be a factor of about 100 and the web crawler is a little bit more selective on what links it follows. Of course the results are still terrible as I have not implement any form of rating of pages yet.
Oh and I created a very simple website that lets me perform a simple search. This is another example of as simple as possible as its output does not even let you click the results and be taken there, that is for the next iteration.
I had a quick look around for python web frameworks as there was not point having the cognitive load of another language without a good reason. Bottlepy looked the simplest to get going as it is a simple file with no dependencies and then I created about 40 line python file including the in-lined HTML for page creation. And I was done. Further refinements will come in later iterations.
It is not publicly available as this is definitely still a for fun not showing type of project.
Comments
Post a Comment