Skip to content

Latest commit

 

History

History
777 lines (511 loc) · 30.2 KB

notes.md

File metadata and controls

777 lines (511 loc) · 30.2 KB
Raw

Overall Important Notes

Release readiness notes

  • Goal for Alpha (?Beta?) release is January 21.

Stuff from FeedWordPress

Ref to feedwordpress.php stuff

  • ln 163 and 164 have a discussion of the problem with WordPress applying filters to syndicated content.

    FWP uses feedwordpress_preserve_syndicated_content (ln 478) and feedwordpress_restore_syndicated_content (ln 493) to bypass this issue. It takes the content received, stores it and then, after all filters are applied, restores the original syndicated content. Something to keep in mind. We may need to do the same thing.

  • Good idea: providing functions that can be used in the post template, make template aware of syndicated content vs non-syndicated? Get item source and include it, make format different depending on source.

    Ex:

    • FWP's template API starts at ln 305
    • Good start with is_syndicated (ln 317)

  • Should we create a similar functionality to FWP's replacing native permalink with RSS content permalink?

    • A function that FWP uses to have the original syndicated post's link used as the post link is syndication_permalink on ln 547.

    • Another: munge_permalinks at ln 1405.

  • Do we want to match categories? Match tags? Yes!

    • Some of FWP's functions start at ln 1422.

  • Should we worry about pings? See function set starting with ln 742.

  • Interesting notes on Magpie and SimplePie use at ln 1544.

  • Do we need the FeedFinder class on ln 13 of feedfinder.class.php? Doesn't SimplePie do that already? I'm pretty sure it does. Should we bother supplying support to pre-2.8 WP with Magpie? Prob. not.

  • It looks like FWP started working on OPML parsing in feedfinder.class.php on ln 238 with _opml_rss_uris().

Zotero Integration

Stuff from MyCurator

  • Perhaps worth considering the API key mechanism?

  • Simple category creation? MyCurator.php at ln 466. Look into wp_insert_term.

  • Looks like it creates a separate database table to store Relevance rating info. The Relevance class is located in MyCurator_local_classify.php at ln 97.

  • It also seems to create a separate table for the list of sources, ln 789 of MyCurator.php.

  • Interesting use of Google Pie Chart API at ln 1085 of MyCurator.php.

Possible Sortable Meta?

Transients API

Looking at ThreeWP Broadcast

  • save_post function at ln 1280 in ThreeWP_Broadcast.php.

    • ln 1319 seems to be where it unsets the comment_count out of the array it is pushing out.
    • If it grabs the post object, don't remove it from the object, how to set the now included comment count as the comment count for the post?

  • the insert post happens at ln 1475.

  • if it is an update, that seems to happen around ln 1468.

  • The problem then... it could possibly be hacked to pull the comment count on pull and on update, but that doesn't really happen enough to be useful. When a post is first submitted, it is unlikely to have any comments. Thap post may never be updated. If that's the case than Broadcast has no method (at least none already built in) to regularly update comments

    • Finding the comment RSS feed and pulling a count of items from that might be a solution, but it is unlikely to hold more than 10 items at once in the feed.

      • So if we used this method, the RSS feed would have to be checked continually and duplicated into the database.

    • The other method would be to, on a cron, check for the origonal blog with the origonal post ID, delve into the database, pull out a count of db entries in the comments table (wp_comments) associated with that ID (comment_post_ID) and read that count back into the theme, perhaps by filtering the comment count function, or something like that, based on if the post is broadcasted.

      • Perhaps can use is_broadcasting function? Unlikely by looking at it in ThreeWP_Broadcast.php on ln 2185.

  • Useful function get_comment_count from wp-includes\comment.php at ln 446.

  • The container that holds a list of blog IDs? blogs found at ThreeWP_Broadcast.php ln 1869.

  • Can use get_post_broadcast_data (from ThreeWP_Broadcast.php ln 2033) on each blog ID? Do I have to rotate through each blog post to check and then send info?

Use WP Cron for pulling posts?

Ref:

Using PressThis button.

  • First 91 lines?

  • wp_remote_fopen is a useful tool to remember. Something to play with later.

  • Looks like it pulls images only when it is an image only page that the bookmarklet is triggered on.

    • Perhaps worthwhile to make it trigger on known img sites as well? Flickr, Imgur?

  • Handling of YouTube and Vimeo occuring at ln 360, fairly limited.

  • If nothing is selected, use readability to pull it all in?

  • On ln 455 are the hidden fields to add stuff to the post.

  • Publish button in the new press this version doesn't do anything in nominations. Should remove anyway.

Styling

  • half width of container
  • 500-600px wide?
  • Larger titles
  • Preview vs Whole showing (jQ to shorten?)
  • Keep the Read More.

Changes to the Under Review nominations section

  • "We also talked about the interface for the "Nominations/"Under Review" section. It would be most helpful for us if this area was for reading and commenting only, without the ability to modify the content of the item. Is it possible to make this more readable, but still allow for comments (through Edit Flow would be fine). Then if an item is Accepted, all content and comments should move to regular post section of Dashboard. If it is Rejected, then those posts could be cycled out of the database over time in order to save space."

Ideas for better visualization of syndicated content

  • We could use a function to include a preview image in both the feed and the nominations page. Perhaps even the post page? We can grab the opengraph image, if that isn't there grab the first image in the content. If that isn't there, a default image or no image?

  • What (if anything) should go in a dashboard widget?

    • Looking at FWP, is it really pulling down the array of Dashboard widgets and adding to them (ln 1186)?! Isn't there a function for that? Like wp_add_dashboard_widget.

  • If there is no featured image, we should build a function to just pull the first image within the readability defined body-content.

  • Be able to star content for a personal read later.

  • Map 'next article' to arrow keys.

Ideas for plugin-modules

  • Auto-tagging using a semantic api.

  • Suggested articles using the Zemanta API.

  • "Inside-out" - let people nominate without letting them in (or at lower user levels).

User feeds

  • Allow users to nominate feeds for addition.

  • Add new subscription box in the feed reader mode.

User control

  • Allow admins to determine what user level is required for each panel of the plugin.

  • If (userlvl == option-stored-userlvl) - display...

  • Allow the info box in under review to display metadata, users nominated, category add/subtract, tag add/subtract, send to draft.

    • Auto tag based on feed folder?

Under review

  • Under review when send to draft - disappear from database.

    • Show hidden stuff as an option

    • Also have an archive button

  • EditFlow style consideration status.

  • Crit path for being able to save:

    • Build the buttons and fields we want.

      • For the fancy wordpress stuff, check out wp-admin/press-this.php ln 447 down.

      • We'll need the same set of post_meta fields for submission as before, to properly send all that meta info on to the draft.

    • Create an AJAX method for capturing the data and saving it (basically what we already did with putting content into a nomination only using a version of the function we created for the saving of a nomination into a post on publish.

    • Have it do something post-'publish'.

Bugs

  • Need some way to have info-boxes not resize their parents, but still work in full-screen mode.

Possible Issues

  • Do we need to worry about updating posts when their origin post is updated? In other words, if the external post is updated should we update the internal post? If we do is there a time limit for this synchronization?

    • FWP deals with this in a number of places

      • feedwordpress_post_edit_controls at ln 805

  • Do we want to assign authorship of a post from the author property of the RSS feed item? Perhaps do that and also pass and display properties for orig author, orig nominated, and editor who published?

    • If so... how to handle inc. authors? A separate options field?

  • If the feed parsing happens on our side, how much functionality needs to be built in for figuring out an RSS feed if it isn't found via SimplePie? Just how good at finding RSS feeds is SimplePie?

  • If the feed parsing happens elsewhere, does the plugin need to have an API key? For each person, each site?

  • A thought? If we want to maintain using the RSS feed to push down from the recommendation engine... why not use the media:rating node in RSS? SimplePie has a function to deal with it. I know it is not what it is made for, but this way we don't have to worry about build a whole bunch of extra stuff to parse the engine's output.

  • Do we want the nominate button to send people to the edit nominations page or allow people to go down a list nominating one item after another?

  • Do we want to try and match authors to existing users on the blog?

  • Should we be importing RSS tags or categories? If we import them into the nomination, should we import them into the post as well?

  • We can add the ability to sort nominations using http://yoast.com/custom-post-type-snippets/ but that's just a straight sort, we really need to sort by date and nominations. How?

  • The last modified column is also really narrow, I'll need to apply custom styling via an attached CSS sheet to that page.

  • Do we want a source link along with an item link? Prob.

  • The refresh button needs someway to stop users from hitting it while it is processing, otherwise it may create doubles.

  • What happens in a large database of posts? Can we split a query up in parts to prevent memory overflow?

  • Error exists at ln 702.

  • Slideshares do not provide good data for the readability to parse. The OG tags for descrip are pretty mediocre as well.

  • Need to insure that all strings are ready for localization. Use WordPress's functionality.

Modules

  • Setup with an init method - provide data: have feed, name, description, slug, img?

  • Have the array merge happen in the base class and have users call it as a register_widget function type thing.

  • If loop added to the module to determine if it is on and off after it IDs and then don't do hooks.

  • If parent::start = false then __construct should return.

  • Instead of forcing the user to build an on/off button into their module, can we automate the module creation?

    • What about building it so it automatically creates the options based on the folder name? The module can then figure out where its relevant option is stored and check it. That way the option and check can occur without the module containing any information about that.

    • The only issue is that we do want some sort of method for providing a module description to users, we can't just cancel the whole init... Instead, set up a function that the base class calls before anything else that contains the 'set-up' information. We just need the user to use that function from the base class and return an array with whatever information we need.

  • Line 62 is where the RSS import module previously ran all content through readability. Let's get rid of that now.

  • Possible to leverage the Readability process to grab real-source authorship when item source is through an aggregation tool.

    • Would need to insert into Readability process or create similar process to search for author data.

    • Could try linking it to a lookup of any Google Author tags on the site or an Open Graph author tag, or both. Not super important but something for the backburner.

The Feedbox

  • Existing AMIs for handling, storing and making accessible feeds?

  • Store it as a public AMI for distribution?

  • Start simple, build up.

  • Needs to run readability on our side as well.

Social Media Integration

Using Google Feed API

  • If we want to include featured images, how best to go out and grab them? Do we even want to try and deal with the resulting (inevitable?) slowdown?

  • How best to set Readability trigger? The current functions to show/hide excerpt/content occur on the show and hide triggers. But selecting is not easy. The selection for the trigger occurs on a.accordion-toggle. The jQuery has to walk up from there to parent .accordion-heading and then up to .span12 and up once more to div.feed-item to find the id which stores the hash id of the feed item.

  • jQuery should select on show the parent id. Which means that (if generalized as opposed to current on-each-item PHP generation) it has to select based on a.accordion-toggle ex: jQuery('a.accordion-toggle).on('show', function () { });`.

  • Actually, looking at the code again it looks like the show and hide triggers occur on that parent div. So I should be able to get the result info (the id) directly. Let's test.

  • To get Readability AJAXed we need to modify the function in rssforward.php at ln 879.

  • To best implement Readability, we need to note if the item is aggregated on pull. We can pull the content of a span via jQuery and feed it to the new readability AJAX to force trigger when needed.

    • Perhaps in the author field?

  • For nominations to work best, they need to be attached to a unique ID. With SimplePie it uses - http://simplepie.org/wiki/reference/simplepie_item/get_id.

  • Clean sweep for renaming to PressForward and making sure that it is ready for localization.

Setting up an editorial 'guest editor' process module.

  • Build out module where you can see all the user stats by admins.

  • Allow non-admin users to request guest editor status

  • When approved by admins guest editors are promoted in user level by WP_cron and at the end of the period demoted by user crons.

Feed Retrieval

  • We need a list of categories that we are going to store.

    • Feed:Title
    • Feed:URL
    • Date
    • Author
    • Permalink
    • Content
    • Tags
    • Word Count
    • Users:read - relationship table - number table?
    • Users:stared
    • Users:nega-stared
    • Item ID
    • Nomination Count
    • ?Source Repeat?

  • Relationships numbers to terms would be a flat data? - store in an array in the plugin/module/core

  • ObjectID, UserID, RelationshipID, Value (user actions table)

    • 38, Boone#, Read(/4), 1

  • Object table (feed items)

    • need functions for creating things.
    • Need various ways to retrieve.
    • Need a custom loop.
    • Need an Item Query with and AND clause added on to retrieving the list of IDs.

  • Feed item metadata table - use exisiting.

  • Look at wp-includes/taxonomy.php

    • ln 2147 wp_set_object_terms

      • pass the slug from insert term, the item object

    • ln 2018 wp_insert_term

      • prefix all the args to insure uniqueness.

    • Pass in the feed item ID (unique table ID) for the objectID - uses numbers only

    • get_object_terms

  • WP Cron - start the feed import process

    • If we are not in the midst of itterating through feeds then...

    • Then stop

    • Then schedule new event from 1 sec before.

    • Iteration is stored as a site option.

    • Then schedule trigger a remote get for wp-cron.

    • Intel about when things update?

    • At the end of the list, delete the iterator options value and reset timer to 24 hours

    • Figure so users can trigger (de-prioritize)

    • You can set up a real cron to hit WP Cron

    • Insure that there is a seperate function for javascript instead of wpcron to retrieve feeds x thru y

    • Would need option for feedcount and feedscounted.

    • Reworking assemble_feed_for_pull

      • Instead of fetching the feed through the whole feedlist, it needs to run the whole process, but on a feed by feed basis. +

    • Pseudocode:

      
create_feed_list
  feedlist
populate_feed_list_option
  feedlist INTO feedlist_sql_option
scheduale_feed_in
  SCHEDULE assemble_feed_for_pull IN 00:30:00
assemble_feed_for_pull
  GET array_of_feed_items
      GET module_x get_data_object
      GET rss-import get_data_object
  CHECK to avoid repeat
  ASSEMBLE into posts
rss-import get_data_object
  GET feedlist_sql_option
  feedlist = feedlist_sql_option
  IF !feedlist_iteration_option {
      MAKE feedlist_iteration_option = 0
  } else {
      GET feedlist_iteration_option
  }
  IF count(feedlist) >= feedlist_iteration_option {
      GET feed OF feedlist WHERE key == feedlist_iteration_option
      IF feed OF feedlist == '' {
          GET NEXT feed OF feedlist
      }
      IF count(feedlist) == feedlist_iteration_option {
          UPDATE feedlist_iteration_option = 0
      } ELSE {
          UPDATE feedlist_iteration_option = feedlist_iteration_option++
          SCHEDULE assemble_feed_for_pull IN -00:01:00
      }
  }
  pf_fetch_feed(feed)
  ASSEMBLE into array
  IF feedlist_iteration_option != 0{
      WP_GET_FILE wp_cron
  }
  RETURN array.

  • Build Under Review, Then WP Cron, then start building the custom taxonomy.

  • Some notes on important options for the iteration process. Note, their functionality has changed over time, so there may be places where they are still being updated even if they don't need to be.

    • feeds_go_switch - The in-process iteration control. This option is designed to prevent the process from ever iterating to get a new feed while it is still trying to get a feed.

      • At the beginning of the entire process it should be 0.
      • When the process begins it is set to 0.
      • When the process ends it is set to 1.
      • When the process is sent to iterate it is at 1.
      • When the entire process ends it should be at 0.

    • feeds_iteration - The step control. This option is how the system tells itself which feed to attempt to retrieve.

      • At the beginning of the entire process it should be at 0.
      • When the process begins it is set to the feed to retrieve.
      • When the process ends it is at the next feed to retrieve.
      • When the process is sent to iterate it is set at the next feed to retrieve.
      • When the entire process ends it should be at 0.

    • iterate_going_switch - The in-process loop control. This option is to prevent the feed retrieval process from multiplying in the midst of the ongoing process.

      • At the beginning of the entire process it should be 0.
      • When the process begins it is set to 1.
      • When the process ends it is set to 0.
      • When the process is sent to iterate it is at 0.
      • When the entire process ends it should be at 0.

    • ready_to_chunk - The external-to-process iteration control. This option is designed to prevent the process from being triggered while it is already running, either by a user or a mis-fired iteration.

      • At the beginning of the process it should be 1.
      • When the process begins it is set to 0.
      • When the process ends it is set to 1.
      • When the process is sent to iterate it should be at 1.
      • When the entire process ends it should be at 1.

Design

Prioritizing CS stuff

Branch Orgs

  • Bugfix branch for minor version changes - ex: 1.5.x

  • Forward looking branch for next version - ex: 1.6.x

  • Master branch = current stable version

  • Moving the logging process to the main process?

  • Moving the schema to the main process?

  • Turn off logging with a constant.

Unit Testing

Prep for reformation

  • Email at end of day stable branch note.

  • Turn on the security.

  • Change everything RSSPF to PF

  • Move to a github group (as opposed to personal)

Build out for trainer:

  • Present the feed data
  • Retrieve the feed data
  • Button to submit data
  • Setting to submit nominations to the trainer automatically or via pop up.
  • Present feed as XML
  • Retrieve feed as an XML
  • Prediction Phase
  • Reaction Phase
  • Return Phase
  • Having the sorter as a self-launching AWS image.
  • Image updates itself from github on startup.
  • Build like the Google API
  • Add nomination text on post not working
  • Not registering sent to nomination from add nomination as nominated in under review interface
  • Pressforward-16.png has bad extra slash
  • Switch from all SVGs to PNG

Classifier more:

  • File must be downloaded
  • Get every item from query from unixtimestamp on forward.
  • Will read in the results from XML file along the lines of RSS Spec. -

Feeds as Posts:

  • At time you record the post set it as child to feed item.
  • Any post can be child to another, regardless of post type.
  • Post children don't take any properties of post parent.
  • post_parent_in - array for query - http://core.trac.wordpress.org/ticket/11056
  • Ex: Set up the query to get feeds tagged wordpress and then get all items attached to those feeds.
  • Duplicate the taxonomy only to drafting (link tax declarations, as a public function, to OPML files?)
  • Parent tags as a separate taxonomy
  • Look into WordPress blogroll.
  • Query feeds with category > get IDs > Query post_parent_in with feed IDs
  • Not a module.

Gen. Fixes

  • pf_simple_html_dom <- simple_html_dom
  • change the script registration to prefix with pf_
  • Check console in plugins that have conflict.
  • grep plugin directory for javascript library names
  • Encode author?
  • Instead of preventDefault try return false. preventDefault on this? Allow clicking on <article> to mark it as read.

UI/UX Change Notes

-    UI change state elements - Views, Sorts, Filters, Folders
- Get feed favicons for foldering
- Need a Help context - in pulldown menu
-    Add Menu
    -    Full Screen
    -    Help
    -    Refresh
    - Pagination/Scroll
    - Nominate This Bookmarklet
- Green - Successful action
- Blue - Something/Content there
- Orange - Debugging Message
- Red - Danger - you are about to do something unreversable
- Yellow - Staring.

Nov 19, 2014 - PressForward: List of potential future priorities

 - Archival function (dump old items to seperate table)
 - Accessible feed list as OPML
 - Ability to subscribe to other sites’ OPML lists
 - ‘Subscribe to OPML needs to work for this’
 - Access WordPress sites’ feeds via WP-API if available.
 - Subscribe to Twitter accounts.
 - Hitch retrieval methodology to Heartbeat API
 - Hitch refresh of All Content and Under Review to Heartbeat API
 - Not really refresh, just adding items to the top on Heartbeats
 - Additional template tags and widgets for exposing nominations.
 - Social sharing buttons
 - Including Buffer - https://twitter.com/Chronotope/status/532609477934342145
 - Draft from All Content
 - Publish from All Content
 - Switch CSS to SASS
 - Bring back the leaderboard.
 - Look into implementing MSET archive attributes
 - https://github.com/PressForward/pressforward/issues/255
 - Detect dead pages and don’t forward.
 - Add site as feed bookmarklet
     - and/or add this as an option to Nominate This
 - Vagrant Package

Ver 4

  • Will create an option that makes Adds Feeds accessible to contributors. Any feeds added will be submitted as Under Review feeds.

  • A shortcode or template tag will make the add feeds form accessible from the front end of a theme for logged in users.

  • PF-API-Phase 1

  • Generalized Post-with-Metadata Object

  • What kind of metrics do we want endpoints for?

  • Nominations

  • Shared "Amplification Effect"?

  • Could also make connection with google analytics

  • Facebook, Share to Email, Post to Tumblr, Pinterest

  • Create a variable that will save all shares.

  • Post from email -- using API and If This then that workflow.

  • Item Read (opened in All Content)

  • Generalized Feed Item

  • PF-API-Phase 3

  • Hook to ingest archive in api but also to trigger readability on the items pulled out.

Extending to other CMS:

  • build out php - easier to take php and separate out the WP dependent stuff. Create a PF specific object
  • custom PressForward options
  • Post stuff to API [is this inbound or outbound?]
  • Self contained Nominate This Bookmarklet

EG

  • Stats API - public facing - data about published posts is public, turn on and off for nomination and item data, default to off.
  • Requests from Partners
  • Request a URL for the PressForward site
  • Check for plugins that use phone home plugins with GA
  • Trigger opt in language on update and install.
  • Any beta functionality set to a particular thread.
  • Capable to edit the source statement from bookmarklet and set default.
  • Panel to add Metadata Fields to the Bookmarklet, with checkbox for adding to draft screen.
  • Allow them to define a meta tag name that it can scrape.
  • Add a feed without adding a post in the bookmarklet.
    • Edit metadata about the feed when you submit it.
    • A Valid RSS feed was found prompt.
    • How to find a valid RSS feed if the system can't.
  • Check the checkbox when a category is matched.
  • Add API examples to the Wiki
  • Twitter
    • Pulling in a link from a Tweet from a user and adding to the metadata a hat-tip name, hat-tip url
    • Pulling in a link from a User's list
    • Pulling in a link from a Twitter Search
  • API Endpoints for Nominations Occurring to send to Slack
  • API Endpoints to push a link to nomination from Slack
  • Unit Testing
    • Make Models happen
  • https://mercury.postlight.com/web-parser/
  • Make it Version 5
  • 5.0
    • Registry
    • Stats
    • Number of Nominations a post has Received
    • Number of Nominations a feed has received.
    • Number of posts shared through social media from PressForward
      • Amplifications per Feed
    • Nominated Data
    • Items Data
    • User Leaderboard
      • Nominations made
      • Save date of nomination with nominating user on post
      • Give me nominations between this period and this period.
      • All the nominations between X and Y and filter by user who has nominated.
  • Ver 5.0 done by 10/31 (Also, API Documentation)
  • Ver 5.1 done by 12/13
  • 5.2 Patch time 3/12
  • 5.3 - 5/15
  • Summer for Twitter
  • Set Up API to take PF site names/urls and put them into a CSV.

Nominate This Mark 2

  • Panels like Press This

  • Deprioritization of main content field.

  • Site options on what pannels appear

  • Add feed with post as part of the pannels

  • Show categories selected including active matches to tags.

  • Allow site-level additional pannels or pannel settings.

  • Add a feed without adding a post. - A switch to move to 'Add Feed' mode instead of 'Add Post' mode.

    • Add a checkbox to say - I'm on the feed, don't attempt discovery.

  • Browser Width to determine bookmarklet width.

  • Comment pannel in the bookmarklet.

  • Featured Image pannel

    • Featured image chooser
    • Featured image removal

  • Smaller title area

  • Options to set site instructions and put that in the bookmarklet? As a pannel for site instructions.

  • Notifications icons in the topbar with

    • Feed Nominations.
    • Unread Nominations
    • ?Comments

  • Move comments to their own table?

  • Metadata icon - https://thenounproject.com/term/metadata/38248/

  • 93 issues to check through

// https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Content_scripts