enhanced functions and final work to do

September 29, 2016, Rolf W. Eckertz
Enhanced functions go beyond the core diary functions.
November 21, 2016 - update on the progress

On the one hand side the aspect of income, expenses and working hours can be extended far more to projekt costing, and many other aspects until taxation on work and travel in Australia.

But the aspects of community oriented functions are much more interesting and challenging:

• themes can be given to diary entries, like politics, economy etc.
• you not only write what you were doing but also write what you think and feel
• and you can comment and vote on texts
• agree
• disagree
• irrelevant
• don't understand
• search functions can support finding themes
• realtime messaging about comments and votes

On the other hand there are some functions still under construction and these should be finished first. So what bothers me:

• make entry panels responsive (done)
• entry of data is necessary - independent of the device, the user has the choice so there may be no restrictions on the user. Tabular entry is a must and for the smartphone there must be an automatic conversion in a responsive way to compensate jQuery Mobile deficits
• table with classic rows works excellent
• jQuery Mobile offers data-role="table" class="ui-responsive" for automatic conversion, it looks not the best, but it's acceptable, just the space between subtables is too big, it comes from a column that is reserved for buttons in the input row, optimization is postponed
• new aspect: so far the elements of dialogues were considered regarding responsiveness, now the general layout of of dialog has to be optimized (open)
• security still has white spots (done, no detail-link for coreader)
• protection of diary entries against unauthorized changes can be done in two ways:
• complicated way: if a user sees input fields, he gets bad feelings if an input is ignored, it's better to have native display than to have protection on input fields, display mode is necessary for co-reading and for "old diary entries". Data in the ui must get an edit mode and a display mode, not very much work, but very important
• simple way: no link to detailed display and edit-panel for coreaders - the simple way has been programmed (done)
• only new data may go to the server, not redundant old data 
• a public reader option will be added in a future release - if that is done, the a publish-date will be introduced, that makes sure, that a user can write the diary during vacation, but the vacation is only visible in the public when he is back at home
• the data model may need some enhancements for that
• opportunity to enter diary entries for days in the past (done)
• the additional entry to dates in the past is provided as special function with a flyin menue in the entry dialog, the datepicker is provided to simplify date entry.
• the datepicker shows the days which already have a diary entry with a special background-color, see http://www.jquerybyexample.net/2012/05/highlight-specific-dates-in-jquery-ui.html and http://www.dotnetcurry.com/jquery/1209/jqueryui-datepicker-tips-tricks, but the right trick came from: http://stackoverflow.com/questions/9023818/correct-way-to-change-specific-dates-cell-background-color-on-datepicker
• new startup for the app - calendar (done)
• the first page still remains the login page, but the second page changes (the app may start with the calendar skipping the login)
• a calendar as startup can show the actual day and month and the days which already have data are marked by color
• the calendar can be used to navigate to the past and to the future, so reminders for coming days can also be entered
• a flyin-menue can give additional choices, so the classical menue is not longer the start
• problems on iOS, first step: enhanced logging 
• copy to the report field (done)
• so far there was a preventDefault on the past action to the text input
• that has been refined, so past is possible
• the formating gets lost, perhaps that will be refined in a later release
• Remember me on login (done)
• a cookie stores the credentials
• enrypted data in the cookie
• cookie is deleted if remember me checkbox is set off
• Rich text in the entry field (done)
• choice of standard tool  for rich text entry
• additional comment-function for co-reader
• ckeditor requires ressources, may be it's better not to allow it on smartphones and only on tablets, laptops and desktop
• realtime messages to the users (done)
• the administrator can enter messages to users, e.g. regarding updates or new functions in the app
• a counter of these messages is shown in the footer
• a special chat dialogue is provided for push messages and chat
• a channel system gives structure to the messages, only certain messages go into the message area in realtime
• the messages are shown on the login page and in the message area
• the user acknowledges that he has read the messages (later, may be)
• initial performance on startup of the browser-solution
• the start of the app gives the first impression, slow start gives a bad impression
• aggregate js files to one file
• compression of the file
• using cache of the browser and controlling the cache from the browser app
• automatic update to the cache with new versions of js files
• persistent data in the browser if the network connection is down
• thats difficult, because the browser has no cross platform storage
• but a fallback for emergency cases is the requirement, not a local database
• cookies and local files are real candidates for the solution
• translation is really an issue
• I really like the idea of international engagement in discussions on one subject
• I made good experience with machine translation
• I learned that text has to be written in a machine translation friendly manner
• realtime feedback by votes and not only by chats (text)
• that's also a personal issue
• votes are an abstract feedback to a subject - easy to aggregate and easy to do
• votes are a good opportunity to express emotions in a constructive manner without offense
• votes are transformed to push-messages in a votemessage channel, this will make it easy to implement
• the vote is transferred as message to the server
• the server calculates the counters
• the counters will be send to the users, not the individual votes
• put all functions into an app (done)
• that was an easy part, the js files are shared

It is not easy to decide priorities, but I can only program one step after the other.  Do I added the priority aspects to the above points.

automatic translation

September 27, 2016, Rolf W. Eckertz, Germany
The basic ideas for automatic translation are remembering text that has already been translated, recognizing text fragments that have to be excluded from automatic translation and offering a correction-option after automatic machine translation.

In the past I told that the principle of a self learning translation.

There are different kinds of text which have to be translated:

• UI-texts of the application
• labels - text which comments input fields and other parts of the ui
• placeholder - text that is shown within input field
• options - text for select lists
• messages - generated from the program, excluding text-fragments that shall not be translated
• hints for the user, help text - that will be avoided, because the app should be self explanatory
• logging data will not be translated
• user-texts
• application specific data entered by users
• diary entries - diary entries can be translated, so friends and co-readers can be international
• comments to diary entries

Regarding the point in time, when the translation is done, there are three choices:

• translation at buildtime - that means
• analyze source code and certain tables and files, extract text to be translated, translate and deploy the matching for the translation
• translation is done with the translation database or by calling the translation API and manuell check and correction
• translation at runtime - that means
• analyze the text that goes to the ui
• translate from the matchings (Translation Database) or call the translation API
• store new translations for later check and correction
• show translated text in the ui
• runtime translation can be used for ui-data and for user-data
• combined solution
• first translation at buildtime
• then translation at runtime, if necessary

Normally translation an runtime is more convenient and during the testing phase it's a good side effect for test coverage and translation coverage. 

Translation is delivered based on the preferred language in the user options compared to the language flag in the application and in the data (texts). It is expected that a user writes in the language he has choosen in the options and that he wants to read the texts in the language. For better efficiency two languages can be choosen for reading.

The following aspects are relevant:

• resource files can be used, not my preference
• string-matching database
• key is language and string, content is target language and translated string
• hashcode-matching database
• for every text a hashcode is calculated
• key is language and hashcode
• content is text, target language and translated text as well as hash-code for the translated text
• exclusion from translation 
• at first the text is examined regarding exclusions
• exclusions are replaced by placeholders, the placeholders are stored with the text fragments to exclude
• the hashcode is calculated with the placeholders
• the target text is found
• the placeholders in the target text are substituted with the stored text fragments
• the final target text can be shown
• that means: before the translation the exclusions have to be marked in the origin text
• hashcode and exclusions with placeholders are a very efficient basis for translation - proved since decades

There are various offerings for webservices for machine translation. Google and Bing surely are big players, but there are also smaller companies like SYSTRAN having good solutions.

Here bing translation based on AJAX will be the first choice with a very attractive pricing - generous free quantitiy per month.

The principle is as follows:

• first check if a text is already translated (remember-function)
• check against translation cache (in memory)
• check against translation database
• if not, the machine translation API is envoked
• the result is stored in the cache and in the database
• the result is flagged as "not yet controlled"
• a dialog is provided to check all machine translations
• correction-option for the translation
• tagging option for the text fragments that should not be translated
• repeat option for the machine translation

Now a certain amount of programming has to be done - and standards like http://i18next.com/ are checked to make the development more efficient
A detail concerns singular and plural respectively pluralization, jed seems to be a good approach regarding that. Professional offerings for translation must be mentioned, like http://locize.com/ - but for starting my non commercial app bing will be preferred.
And I go into https://medium.com/@jamuhl/translate-my-website-please-732ddb622cba#.paw31btxw and formatting may not be forgotten: http://formatjs.io/guides/ and especially http://formatjs.io/guides/message-syntax/ 

The qualtity of machine translation depends on the "machine friendliness" of the text. There are some rules which support machine translation - and they usually enhance the readability for humans too. Some recommendations from http://translation-blog.multilizer.com/guidelines-for-writing-text-that-machine-can-translate-better/

• write short sentences
• write full, grammatically correct sentences
• use common vocabulary
• avoid words that have several meanings

These recommendations should be obeyed by programmers too, who write help-texts, messages or documentation.

Dynamically load js files and scripts

Dynamically loading code is the core of automatic updates and a good opportunity to reduce loading times for the browser.

At first there is an analysis of the most used pages for the "normal user", these pages are loaded at start of the browser-app. The pages which are rarely used by "normal users" and the pages reserved for administrators are loaded on demand.

Loading is done by ajax.

This way a caching in the browser is not necessary.

Regarding apps the ajax-option to load js code is used for updates. Hashcodes for the js files are used to check, if updates are needed - same principle as with GIT.


http://demos.jquerymobile.com/1.4.4/pages/
https://api.jquery.com/jquery.getscript/ https://www.tutorialspoint.com/jquery/ajax-jquery-getscript.htm gives a good introduction.

The strategy is as follows:

• initially the main js modules are loades from index.html
• the additional js modules are loaded on demand during execution

Each module corresponds to one page. So far it is checked, if a page is already initialized on invocation, now there is an additonal check, if the page module is already loaded and loading is done accordingly.

The loading is done with $.getScript, the sample in tutorialspoint seems to be good:

               $.getScript('result.js', function(jd) {
                  // do something
               });

The first test is done with a minor function for admins.


The following problems have to be solved:

• the menu-control uses href and the hashcode of the target page, that does not work, if the target page is not yet loaded and initialized. The solution will be to capture the click-Event on the menue point and then do the loading as well as pagecontainer change to navigate to the new page, the preprocessing for the new page is done in the control logic as it is

So this is the code:

        $('<a/>', {

            id: "loginControlLink",

            href: "#",

            html: "Login-Kontrolle"

        }).appendTo(

            $('<li/>', {

                click: function (e) {

                    e.preventDefault();

                    try {

                        if (!$("#loginControl").length) {

                            var url = "js/uiloginControl.js";

                            $.getScript(url, function (data, textStatus, jqxhr) {

                                // console.log( data ); // Data returned

                                // console.log( textStatus ); // Success

                                // console.log( jqxhr.status ); // 200

                                uiloginControl.init();

                                $.mobile.pageContainer.pagecontainer("change", "#loginControl", {

                                    transition: "flip"

                                });

                            });

                        } else {

                            $.mobile.pageContainer.pagecontainer("change", "#loginControl", {

                                transition: "flip"

                            });

                        }

                    } catch (err) {

                        uihelper.putMessage("Kein LoginControl möglich:" + err.message, 3);

                    }

                }

            }).appendTo("#navpageListe")

            );

The old controller-logic remains:

                        if (link === "#loginControl" || toPage === "loginControl") {

                            console.log(">>>loginControl");

                                uiloginControl.beforechange(event, ui, function (err) {

                                    callback(null);

                                    return;

                                });

                        }

This technology can be used with a remote server or locally, if an app already has downloaded updates before.

travel app community

September 2016, Rolf W. Eckertz

The development so far was concentrated on the individual usage of the travel app, "work and travel diary", and co-reading in the family. The feedback and some market research led to the following ideas:

• the app must be perfect before it is presented to the public in an app store
• you never have a second chance for a first impression
• fast start 
• excellent performance
• ease of use
• no user manual must be necessary
• no user help must be necessary
• common user expectations must be fulfilled
• secure, secure, secure
• encryption of data stored in the database
• encryption of data transfer
• user identification and authorization
• no deletion, no change, only adding of data
• multilingual and global
• shipment in English and German language
• working with different time-zones
• working with different country standards
• integrated translation services
• always on - so far a login is necessary, that will be changed:
• a cookie will hold an endless session
• the user can finish the endless session (button)
• the app version will be always on, the browser gets the new functions
• save all entries
• perhaps a cookie is a good idea to hold the last entry even if there was no connection and the user did not click a save function
• viral community building
• you need an invitation from a user to become a new diary user
• an invitation is a key that is generated by a user
• the key may be transferred by mail, Facebook-Chat, whatsapp, Skype or other services
• to simplify the invitation the address book of a smartphone or pc can be accessed (will not be possible on all systems)
• within the diary community an email address is unique, it can only belong to one user
• but one user can have two or more email addresses, because he is free to change the provider over time
• an invitation may or may not include the co-reading option

When this is delivered, then the next step can be done: enhancing the travel app to a community solution. What does this mean:

• users can define a community diary
• users can invite other users to take part in this community diary
• users can only read a community diary if they add information into this community diary - "do ut des" or "quid pro quo" are the principles, od: "no output without input"
• all community members can get the updates of others users of the community with a realtime push-service

What a challenge for the future

co-reader support

September 14, 2016, Rolf Eckertz
Co-readers are users that are allowed to read diaries of other users, who have given them the according access rights.

A user may declare certain aritcles or data as private, then no other co-readers have access to these data.

The data-categories are:

• diary reports
• expenditure-data
• income-data
• (work-) time-data
• geolocation information

The system provides the following functions:

• Invite other users as co-readers to your diary
• enter invitiation in the diary application
• choose menue point
• dialogue function
• entry field for email address of co-reader
• formal check of email address
• entry field for additional text for the user who shall be invited
• on click - AJAX-request to the server
• send email address and additional text to the server
• server stores the data
• server calculates onetime handle
• server sends response to client
• client starts local email application and passes address, subject and body for the email (some email programs must be started manually in advance)
• some information in http://stackoverflow.com/questions/36667980/can-i-open-a-mailto-link-in-gmail-and-have-it-sent-without-clicking-send 
• window.location.href="mailto:myemail@gmail.com?subject=My+great+email+to+you&body=" + text;  
• email is send to the co-reader
• co-reader can register as reader to a diary
• click on link in the email or copy the link to the url of the browser and send it to the server
• if everything is ok, the co-reader registration is done automatically and the browser starts the login, otherwise an error message is displayed and no co-reader registration is done
• the co-reader can read the diary of other people according to the registration and acknowledgement

In future releases there may be additional features regarding granular security on diary-entry basis, some entries then may be declared strictly private or groups of co-readers may be supported. At the time there is no plan regarding the realization of these additional features.

uniform messages to the user and standard functions

Updated: November 8, 2016, Rolf Eckertz
The layout of the footer must be changed dynamically in same applications, a test with table is very promising, test in the browser are passed, tests with app mode have to be done.
Updated: October 28, 2016, Rolf Eckertz
Check if function exists and layout of navbar

There are two kinds of messages:

• messages that are a reaction to a user-input
• messages that are a reaction to things going on in the background

Messages are shown in the footer.area. In some cases the footer area also has to provide special navigation buttons for paging or scrolling, when swiping and mouse on scrollbar are not feasible.

That is why the footer area is constructed as follows:

• first line of the footer
• last message
• click on this message navigates to the display of the message buffer
• Problem:
• when used as App with Emulator of Intel XDK the formatting is not as expected, the text is presented in the same line as the buttons
• so: on Smartphones the text is NOT presented in the footer, an alternative is checked, my be a temporary popup will dd
• second line of the footer, button area with five buttons
• outer left button: navigate back 
• optional depending on application
• click calls standard-function-name "goprevious"
• inner left button: status indication server-connection
• color green - connected
• color red - connection got lost
• color yellow - connection is active (AJAX is executed)
• click calls display of system values
• middle button: number of messages
• counter
• in the future additional counters may be provided
• click calls display of messages in separate page
• inner right button: refresh button
• click neutralizes the input and makes no update (optional depending on application)
• outer right button: navigate forward 
• optional depending on application
• click calls standard-function-name "gonext"

Technical implementation:

• uihelper.putMessage(text[, severity]) - the central function
• put a message to the message queue (userMessages = []) with message and severity
• display the message in the activePage
• a maximum of 100 messages are cached, no persistence
• the message display area in the footer of the activePage is allocated, if it not already there
• uihelper.showAllMessages() - show all Messages, maximum 100, activated on click to menue point
• uihelper.refreshMessage() - show last Message on new page after change and transition
• function is called when new page is shown ("beforechange" function)
• function builds the contents of the footer
• the standard-function-names are checked if they exist on the page, if yes, then the navigation buttons are shown and the "refesh" button respectively - if no configuration parameters were passes
• the message-queue is stored in uihelper userMessages, access only via api 

Functions have to be checked dynamically, see http://stackoverflow.com/questions/8592047/check-if-a-function-exists-with-its-name-in-a-string

if (typeof window[strOfFunction] === "function") {
    // celebrate
    //window[strOfFunction](); //To call the function dynamically!
}

and

if ( eval("typeof stringFunction === 'function'") ){ /*whatever*/ }

There were several problems in the layout of the footer. Finally the following solution was found:

• for the browser
• the footer has a text-line and a navbar with five buttons
• the layout has a scrollbar beside header, content and footer, it the content needs it
• for an app with Intel XDK
• the footer cannot have a text-line, because it is presented beside the navbar and not above the navbar
• the five buttons in the navbar must be calculated in width with px and assigned to .ui-block-a to .ui-block-e, then the buttons have equal size and cover the whole width of the footer
• the scrollbar is presented aside of the content, it does not show beside header and footer

        if (appMode && appMode === true) {

            $("#" + actPageId + "navbar").navbar();

            var wnb = $(actFooter).width();

            var wne = (wnb / 5).toFixed() - 3;

            $("#" + actPageId + "navbar  .ui-block-a").width(wne + "px");

            $("#" + actPageId + "navbar  .ui-block-b").width(wne + "px");

            $("#" + actPageId + "navbar  .ui-block-c").width(wne + "px");

            $("#" + actPageId + "navbar  .ui-block-d").width(wne + "px");

            $("#" + actPageId + "navbar  .ui-block-e").width(wne + "px");

        } else {

            $("#" + actPageId + "navbar").navbar();

        }

The code has the following points:

• appMode is set in the application, if true it's app with Intel XDK,  if false it's node.js and browser
• actPageId is the string with the id of the activePage
• "navbar" is the suffix for the id of the navbar

The click-behaviour was strange: when you click in an empty area of the content, where no click-event is defined, then the header or the footer of both hide and show. This behavior makes sense when you want to show pictures fullscreen. This behaviour is prevented by global configuration:

$.mobile.toolbar.prototype.options.updatePagePadding = false;

   $.mobile.toolbar.prototype.options.hideDuringFocus = "";

   $.mobile.toolbar.prototype.options.tapToggle = false;


uihelper.refreshMessage (link, toPage) is only called by nta1010login and the central "controller" logic which is more a central support for navigation. refreshMessage is executed before the new pages are activated by the xxx999yyy.beforechange function that a standard page has to provide. That means:

1. the new page is prepared to show and set activePage automatically by the navigation
2. the refreshMessage is envoked generically and it changes the footer of the page
3. then the page-specific beforechange function is envoked to prepare the contents of the page
4. and this gives the opportunity to add specific information to the page in beforechange - and the layout of the content can be adapted accordingly