Paul Hammant's Blog: Scripting AmericanExpress.com To Monitor Temporary Charges
Life in the US requires a credit card. You should ignore the hidden price increase to cover the 2-3% commission that the credit card issuer take, and focus on the convenience.
Temporary Transactions
For some categories of purchase/charge the transaction goes on your account as a “temporary” transaction. These could be known as “pending” or “hold” transactions too. Say you’re in a restaurant, and use your card to pay - the pre-tip amount gets posted straight away. Some days later that is fixed to include the tip, and it gets posted to your account.
While you can be notified of the temporary posting, there’s no automatic way to be notified of its conversion to the one posted on your statement. Not for Amex, Citigroup or Chase cards that is - I don’t know about the others.
You see, you should compare your receipts to the the posted amounts to prevent mistakes and tip fraud.
Monitoring Temporary Transactions
With American Express, in their web-app, you can click to receive a follow-up notification when the transaction becomes final. Having to log in to a website for every pending charge is a bit cumbersome though, so I wrote a Selenium script and put it on Github. Inlined here, too:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import ElementNotVisibleException
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import TimeoutException
import sys
# WebDriver has no element.remove() function, but can gain it via execute_script()
# -------------------------------------------------------------------------------
def kill_element(element):
driver.execute_script("""
var element = arguments[0];
element.parentNode.removeChild(element);
""", element)
driver = webdriver.Firefox()
# Get to the Pending statements and activity page
# -----------------------------------------------
driver.get("https://www.americanexpress.com")
driver.set_window_size(1400, 3500)
driver.find_element_by_id("Username").send_keys(sys.argv[1])
driver.find_element_by_id("Password").send_keys(sys.argv[2])
driver.find_element_by_id("loginLink").click()
driver.find_element_by_id("MYCA_PC_Statements2").click()
WebDriverWait(driver, 3).until(lambda driver : driver.find_element_by_id("pendingHdr")).click()
# Create a bunch of space so that ElementNotVisibleException doesn't happen quite as much.
# ----------------------------------------------------------------------------------------
try:
WebDriverWait(driver, 3).until(lambda driver : driver.find_element_by_id("graphContainerClose")).click()
except NoSuchElementException:
pass
except ElementNotVisibleException:
pass
elems_to_delete = ["pendinghelp-tbody","tbllinks","proc_total","iNavMainContainer","iNavWrapper","cmSnapshot","footer", ".toolbar-wrapper.AXP_iNSlide"]
for to_delete in elems_to_delete:
try:
kill_element(WebDriverWait(driver, 2).until(lambda driver : driver.find_element_by_id(to_delete)))
except TimeoutException:
try:
kill_element(driver.find_element_by_css_selector(to_delete))
except NoSuchElementException:
print "Element not removed: " + to_delete + " - not found"
except ElementNotVisibleException:
print "Element not removed: " + to_delete + " - not visible"
# Loop though pending payments clicking the "mark for monitoring" widget if it is not already set
# -----------------------------------------------------------------------------------------------
tds = WebDriverWait(driver, 3).until(lambda driver : driver.find_elements_by_xpath("//table[`id='listData']/tbody[contains(`class,'colDesc')]"))
for ix, td in enumerate(tds):
attribute = td.click()
try:
WebDriverWait(driver, 3).until(lambda driver : driver.find_element_by_id("unMarked_" + str(ix))).click()
WebDriverWait(driver, 3).until(lambda driver : driver.find_element_by_id("markCharge_" + str(ix))).click()
print "Charge " + str(ix) + " marked for monitoring "
except NoSuchElementException:
pass
except ElementNotVisibleException:
pass
td.click() # close that flyout again
kill_element(td.find_element_by_xpath("..")) # create more space towards the top of the viewport
driver.close()
The Amex page was not easy to script - there is quite a bit of repaint that goes on as you click around, meaning a lot of ‘wait-for’ is required. There is also a lot of scrolling required by the user to get to the part of the page you want to see or click on, and rather than try to simulate the scrolling, I chose to remove parts of the page, allowing the pieces I wanted to interact with to be closer to the top margin. Thus this…
… is reduced to this at the time of the “mark this charge” click …
The script takes about 30 seconds to run. If not on your laptop, maybe you could put it on a box in the cloud so that you can run it daily.
Requests to Amex
Product Owners
Please make a “monitor all pending charges” setting that has a lasting effect. That would mean the need for this script goes away.
It would be great to able to add another “user” with lesser privileges just to read card usage information, that doesn’t require a social security ID as you have it now, as robots are not humans. We’re in the robot era, don’t you know.
Developers
If you’re going to change the HTML for the page, can you issue a pull-request so that I can easily keep up.
Can you lift the password limit too, please? -
Requests to other card issuers
Consider making “monitor all pending charges” a thing in your portal. Amex holds #1 position amongst card issuers and beating them on features should be part of your approach.