Reduce Technical Debt

Hi All,
Overview
For the uninitiated “Technical Debt” is a term where you are having to repeat yourself multiple times on coding, so for example logic changes where you have had to duplicate the widget/favourite, and you have to adjust multiple dashboards to support this one change. For simplicity I am going to refer to a widget/favourite as a “dataset” to make life easier below.

Why is it important?
Simply put, you are not honoring “one version of the truth”. It is important all favourites, dashboards and databases that changing one thing can support all of them. So buckle up this spans several areas! Here are my intial thoughts that I would appreciate Phocas to mull on (further down suggestions on how to fix):

1. Defined Periods.
I have raised this previously but will raise again here. Twice a year I have two major pieces of work to perform on the date ranges. Specifically on reporting periods, these typically are excluding the current month, so the business can review all transactions up to the prior period. I have fiscal and calendar reporting periods.

So take a typical calendar year. In January its should be the entire Jan - Dec of the prior year. In Feburary it should just be January 23 with the end date Jan23 (offset to month -1). 11 Months out of the year this works fine, but every January I need to change the defined periods as the logic would be Jan 23 to Dec 22 (month -1). This is an invalid date range and this impacts multiple dashboards. More over this impacts around 7 databases. I need to go into each one and adjust the defined periods. Now times that by 2 as have the same issue for fiscal reporting periods. Full breakdown can be found here: https://pug.phocassoftware.com/t/master-defined-periods
2. Widget logic.
As mentioned by @Terry.Fung in an old post already: Copy Widget Feature - add to future release.
We have several dashboards where we rely on some key datasets which repeat on all dashboards (I’ll come back to that shortly). Currently we have users clicking analyse on these key datasets and saving as a favourite of their own. Although I think this is great, I also consider this VERY wrong! We have real life examples of people utilising a favourite of a widget which was a copy a long time ago and since it was a custom widget the logic has changed over the years and thier favourite was never updated. This then causes issue as their version is not the same as production due to the changes over the years.

Back to the dashboards, we in IT have to manage the 6 dashboards where this singular dataset is used, any changes means we end up having to make 6 dashboard adjustments… Moreover keeping track of the changes becomes very difficult over time. I.e. is this dashboard adjusted? how about the other ones? This is just one example, we have shared datasets all over the estate.
3. Controlling the overall dashboards data.
I raised this before but will add it again. Imagine you have a dashboard with 20+ widgets. The dashboard is pretty much a copy of your main sales dashboard. The only reason for having a clone is becuase you need to apply a cross spectrum filter to exclude certain items. Just for arguments sake you need it to exclude the customer of Amazon but show everything else. That would mean you would need to edit 20+ datasets to filter out the records. There are workarounds which are to place another widget at the top as a form of filter so you would focus on everything BUT Amazon. However this is a simplistic example and ours become more complex. This greatly increases technical debt as I now need to change logic changes on two dashboards if anything changes. For more detail please see this: Dashboard master filters

HOW TO FIX?
I am sure there are other technical debt style elements that you may think of, however these are my biggest ones. Now on to how they could be potentially fixed…

1. Defined Periods
If you read the other post I made (here), you can see that we make associations in the database defined periods screen to some master defined periods. This then would mean we have one place to control all databases . I.e. I change one date range and all databases (where defined) would then start using this. All the technical debt is gone in an instant.
2. Widget Logic.
This is more tricky and would split this into two avenues:
a) Dashboards
b) Favourites.
With dashboards it would be aventageous that we have the concept of a library. This library would house the standard goto datasets that we would need. This would mean adjusting the library version would instanteously fix all the dashboards rather than having to apply the same logic one by agonizing one.

With favourites, thats a tougher cookie but have a developer spin to make this very easy. Checksum the query! for those not in the know you can put a checksum which is a unique number based on the contents of the SQL driving it (in this case its a widget but same difference). As an aside barcode scanners have check sums, thats why occationally when they dont scan you get “unexpected item” errors this is becuase the number does not equal the checksum (same with zip files!). I propose the same here. User copies the widget, saves it as thier own. This favourite will have a reference to the original widgets checksum. If any change occurs to the dashboard widget the checksums will no longer match and thier favourite displays a warning that thier widget is no longer up to date. If the user chooses to continue using it, its on them, not the problem of the system. Here’s a mockup:


3. Controlling the overall dashboards data
The post already (here) has the potential solution but unfortunatley got sidelined with URL manipulation questions (which is probably the easiest solution Phocas could implement!) Basically clone the dashboard then apply a master filter at the top level which would mean only one adjustment made for the dashboard would be applied to all widgets.

I will get off my horse now as it was getting high :wink: In all seriousness, the usability of Phocas is its greatest strength but is also a weakness. The above I believe retains the freedom but provides a better framework to control the structure and give back much needed time to the admins of the system so we are not chasing our tales…as much.

Look forward to anybodys thoughts on this. I know its a long one, but much overdue knowledge dump from me :slight_smile:
Regards,

Jon

3 Likes

@JonKemp, you deserve a free lunch, drink, or both for documenting this. Cheers and gratitude to you. :beers: I echo your thoughts and share your pain. Any time you want to get on your horse, I’m here to listen. #EndTechnicalDebt

RE Periods: There were multiple reports/dashboards on our site with issues caused by the transition into a new year/period. If not your suggestion, there should be a Phocas engineered solution for this. Jumping through hoops to shuffle dates around, or waiting for the first month to be over is sad.

RE Widgets/Copy: if the above is sad, this is just bad. In addition to all of the other reasons you’ve documented about this, I’ll add that I’d prefer not to have extra ‘favorites’ saved on my site just so they can be used as a way to copy/duplicate items on a dashboard. You said, “Imagine you have a dashboard with 20+ widgets.” I don’t have to imagine, I do… Here to, if not your suggestion, there should be a Phocas engineered solution. Dashboards are POWERFUL when done right but, there are definitely times I’ve scaled back because of these limitations.

2 Likes

Wow - thanks for all of this and apologies for not getting back sooner (I was on leave). I’d love to unpack some of this in a proper session, I’ll reach out to organise a call.

Hamish (Product Manager for the Grid/Query pages here at Phocas)

@hamish.keith - Replied to your emailon 2023-02-23. Please let me know if you recieved.

We’re working on Point 1 right now @JonKemp .

1 Like