Knowledgebase
Knowledgebase: PaperCut > APIs
External Device Test Cases
Last modified on 12 October 2012 11:43 AM

The following test cases are used to test, validate and/or certify PaperCut copier/device integration. The use-cases and tests apply to both PaperCut in-house developed copier control solutions (such as embedded solutions) as well as solutions developed by 3rd parties using PaperCut's Copier Control Terminal API.

3rd party implementations must complete these test cases before being certified by PaperCut. The 3rd party should complete the tests recording the results. Where the implementation does not comply, an explanation for the non-compliance must be provided. The testing procedure below is thorough and detailed. Testers should look at dedicating at least one full day to this process and its documentation.

Test are designed around "real world" use cases and are outlined in detail below.

Test Cases

Test Case 1: Student user

A student is restricted in the amount of printing and copying that can perform. This user will not have access to shared accounts. They can only charge to their personal account.

Setup

  • Setup a student test user in PaperCut
  • Set the user to be "restricted"
  • Set the device copy cost (e.g. $0.10 per page)

Tests

a) User has credit balance:

  • Allocate the user a small balance. E.g. $2.00
  • Login as the student user.
  • Perform some copying (or scanning/faxing if supported)
  • Verify that the job is logged correctly in PaperCut.

b) User has no balance:

  • Set the user's balance to zero.
  • Try logging in as the student
  • User should be denied copy access with an understandable message (insufficient funds).

c) User with small balance (zero stop):

  • Set the user's balance to a small value. E.g. $0.50
  • Login as the user
  • Perform a copy that will cost more than their balance
  • Confirm the copy job is stopped when the user's balance is used up. NOTE: Some devices can stop the job before the job starts, others might stop the job part way through.

Notes for ideal implementation:

  • For "restricted" users, their account available balance should be displayed to them after successful login or throughout copier usage.

Test Case 2: Teacher user

A teacher has access to charge copies to two faculty accounts "Science" and "Maths". They can also charge printing to a personal account which is unrestricted.

Setup

  • Setup a teacher test user in PaperCut
  • Set the teacher user to "unrestricted"
  • Change their account selection settings to "Show the Standard Account Selection Popop" and enable "select shared account with list" only. Disable the "personal" and "shared account with code" options.
  • Create two accounts "Science" and "Maths" and grant this user access to these accounts (through Account->Security).

Tests

  • Login to the device as the teacher user.
  • The user should be prompted for the account to charge their copying. They should be able to select the accounts from a list. The list be:
    • Personal Account
    • Science
    • Maths
  • Once the account is selected perform some copies.
  • Confirm that the job is logged correctly in PaperCut and was charged to the selected account.
  • Repeat this test selecting each account option in turn. Confirm jobs are logged correctly.

Notes for ideal implementation:

  • When users select "unrestricted" accounts the account balance is not relevant and should not be presented to the user. If something is to be displayed it should be "unrestricted".
  • When users select "restricted" accounts the account balance should be displayed to the user.
  • When users have a small number of accounts then it's preferable to minimize the number of steps. It is not appropriate to provide an "Account search" option when the list is so short. When there is a small number of accounts it is recommend to simply display them in a selectable list format.
  • When a "default shared account" is set for the user, the user interface should change to make this account easy to select with minimal steps/presses. i.e. it might appear at the top of the account list so it can be selected with one click.

Test Case 3: Teacher user (pin/code account selection)

In this organization teachers select faculty accounts using PIN/codes and continue to use this convention at the copiers. They are also not allowed to charge copying to their personal account and must select a shared account via a code.

Setup

  • Setup a teacher test user in PaperCut
  • Set the user to be "unrestricted"
  • Change their account selection settings to "Show the Standard Account Selection Popop". Disable the "shared account with list" and "personal account" options. Enable the "shared account with code" option.
  • Set the PIN/codes on the previously created accounts as follows:
    • Science: set code to 1111
    • Maths : set code to 2222

Tests

  • Login to the device as the teacher user.
  • The user should be prompted to enter the account PIN/code.
  • Select account 1111 and perform some copies
  • Confirm that the job is logged correctly in PaperCut and was charged to Science.
  • Repeat the proceeding two steps for account 2222.
  • Repeat for account 3333 (invalid account). Confirm that a suitable error message displays.

Test Case 4: Corporate user

The corporate user has their copies tracked, but they are not restricted from copying in any way (tracking/auditing only). They do not have access to charge to Shared Accounts. All jobs are automatically charged/allocated to their personal account.

Setup

  • Setup a corporate user in PaperCut (e.g. coperateuser)
  • Set the user to "unrestricted"
  • Set their "Account Selection" settings to "Automatically charge to personal account"

Tests

  • Login to the device as the corporate user
  • After login the copier should be enabled for use.
  • Perform some copying.
  • Confirm that the job is logged correctly in PaperCut.

Notes for ideal implementation:

  • In this scenario the objective is to grant the user copier/device access with as minimal user interaction as possible. The aim is to keep user steps to a minimum.

Test Case 5: Professional user (Engineering Company)

An engineering company allocates all printing and copying costs to client and project accounts. With over 5000 accounts to choose from they need to find the account efficiently via keyword search (search and review). For example, list all accounts and sub-accounts for "acme inc".

Setup

  • Configure the "[Template Account]" to grant access to the "[All Users]" group. This ensures all users have access to all new accounts created.
  • Use the bulk account import feature to import 5000 shared accounts. TIP: Use Excel to create a spreadsheet with 5000 accounts named: account0001, account0002, account0003, etc. Then save the spreadsheet as a tab-delimited file to upload into PaperCut. See http://www.apms.com.hk/product/papercut-ng/manual/ch-shared-accounts-import-update.html for information on import file format and procedure.
  • Setup the professional user's "Account Selection" settings to be "Show the Advanced Account Selection popup" and disable the "Allow to charge to their personal account" option.

Tests

  • Login to the device as the professional user
  • The user should be prompted for the account to charge their copying. They must have:
    • the option to select account by pin/code
    • the option to select accounts from a list.
    • And importantly, the option to search for the account by name
  • Select the "Search" option and try the following searches:
    • Search for "111". This should return about 14 accounts if the accounts were created as above.
    • Search for "1234". This should return a single account "account1234".
    • Search for "acc". This will return all 5000 accounts. The device should handle this gracefully by paging through the results (not retrieving all results at once).
  • Select an account and perform copying
  • Confirm that the job is logged correctly in PaperCut and was charged to the selected account.

Notes for ideal implementation:

  • When a user has access to many accounts it is of little use to display a very long list. The user should first be presented with the option to choose how to search for an account (e.g. by pin/code by name, etc)
  • When performing the account searches, a reasonable amount of account results should be requested (e.g. only 100). If the user pages through all these results the device can then request additional results.
  • The user should be offered a method to scroll through the list of accounts.
  • When a "default shared account" is set for the user, the user interface should change to make this account easy to select with minimal steps/presses. i.e. it may appear as an option in the same screen as where the account search is provided.

Test Case 6: User with New Card

Users use swipe cards for authentication. The user is issued a new card and they should be able to associate the card with their user account at the device (self association).

Setup

  • Remove any previous card number from your test user in PaperCut. Do this by selecting the user and clearing the "Card/Identity Number" field.
  • Make sure that the test card number is not associated with any other users.
  • In PaperCut configure the device as follows:
    • Enable card authentication.
    • Enable the "Enable self-association with existing user accounts" option under card authentication.

Tests

  • Attempt to login by swiping the card
  • The device should inform the user that the card is not known to the system, and ask if they wish to associate the card with their account (e.g. Yes/No).
  • If the user proceeds the user should be prompted for their username/password. Enter a valid username and password. The device should indicate that association was successful.
  • The device should then take them back to the initial login screen to re-swipe the card.
  • Repeat these test and ensure an invalid username/password does not allow association and feedback is appropriate.

Notes for ideal implementation:

  • When a card is not known to the user the user should be prompted whether they would like to associate a card. This should not happen without the user accepting this step.

Test Case 7: Secure Print Release

An organization uses secure print release to hold jobs in the print queue until the user releases the jobs at the device. Note: This feature also applies to find-me printing (aka Follow-Me printing).

Setup

  • In PaperCut, enable the "Hold/Release Queue" on a test print queue.
  • For the device, enable the "Print Release" option
  • Select the print queue configured above under the "Displays jobs for release from the selected queues" option.
  • Ensure the "Automatically release user's jobs upon login" option is disabled.
  • Print some test jobs to the queue and confirm the jobs are listed in "Printers->Jobs Pending Release". If the jobs are listed here they will be ready to release on the device.

Tests

  • Login to the device as the user that printed the jobs waiting for release.
  • The user should be prompted to select to use the copier or release print jobs.
  • When user selects to release print jobs they should be displayed a list of jobs to release.
  • As jobs are released they should be printed and logged in PaperCut
  • If jobs are cancelled, the job should not print.

The ideal implementation does depend on the amount of space and GUI options available on a given device.

Notes for ideal implementation:

  • When giving the user the option to select between copying and print release it is recommended to display the number of currently held jobs to the user. E.g "Print release (3 jobs)"
  • Users should have the option to both print and cancel jobs awaiting release.
  • There is a lot of information about the jobs being held (e.g. doc name, user, client machine, time, pages, cost, etc). Ideally all these attributes would be visible to the user, but on constrained devices this may not be possible. The most important attribute to display for print release is the document name.

Test Case 8: Secure Print Release (auto release enabled)

An organization uses secure print release to hold jobs in the print queue. When the user logs into the device they want all jobs to be released automatically upon successful login.

Setup

  • Setup PaperCut as per Test Case 7: Secure Print Release.
  • In the device configuration enable the option "Automatically release user's jobs upon login". This can be found under the device's print release settings.

Tests

  • Login to the device as the user that printed the jobs waiting for release.
  • The login will trigger the jobs to be release. If jobs have been released the user should be informed by a brief message.
  • The device should then take the user through the standard copier process and not display an option to copy/release because the user's jobs will already be released.

Notes for ideal implementation:

  • The message displayed on login if jobs are released should ideally indicate the number of jobs released.
  • If multiple jobs are held for a user but not enough to release all the jobs, user should be given an option to select the job(s) they want to release.

Test Case 9: Lost card (alternate login methods)

User in this scenario normally authenticate with cards, however users should also be able to login with alternate login methods. This is useful when user has misplaced their card (e.g. left their card at home). An alternate login method, all be it less convenient, will ensure the user still has copying and secure print release access.

Setup

  • In PaperCut enable both the "swipe card" and "username and password" authentication.
  • Make sure your test user has the card number set to the number of your test card.

Tests

  • When the user approaches the device they should be able to login with either card or username/password.
  • Test that the user can successfully login with a card.
  • Test that the user can successfully login with a username and password.

Notes for ideal implementation:

  • The user should be able to log in with a card without pressing any keys on the device. i.e. they should not have to first select "card authentication", nor have to push "Enter/OK" after a card swipe.
  • The interface should guide the user to the options available. The interface will depend on the capabilities of the device. On a limited display device the screen might show "Swipe card or press X to enter username/password". On a graphic/touch screen the alternate login options may be presented as buttons.
  • The interface should where possible support all authentication methods:
    • Card Swipe
    • Username and password
    • ID and PIN

Test Case 10: Two Factor Authentication

To prevent unauthorized card use when a card is stolen or lost, organizations may chose to implement two factor authentication (something you have plus something you know). In PaperCut two factor authentication is implemented via a numeric card PIN.

Setup

  • In PaperCut, enable the "Swipe card" authentication for the device.
  • Enable the "Require PIN" option under the "Swipe Card" option.
  • Setup the test user:
    • Ensure the user has credit/balance on their account.
    • Ensure a test card number is entered in the test user's "Card/Identity Number" field
    • Ensure the the "Card/ID PIN" field is clear/empty. This will cause the device to request a new PIN on first use.

Tests

Test for user without a PIN (define new PIN on first-use):

  • Swipe the test card at the device.
  • The user should be prompted to enter a new PIN. They should also be asked to confirm the new PIN to ensure it was correctly entered.
  • The user should be informed that the new PIN is assigned and device's access granted.

Test for user with a PIN:

  • Now that the user has been assigned a PIN (in the previous test), swipe the card at the device again.
  • The device should prompt for the PIN number.
  • Enter a correct PIN number and ensure device use is granted.
  • Enter an invalid PIN number and ensure devices access is denied.

Notes for ideal implementation:

  • Where possible the user interface should adapt based on whether a PIN is required.
  • Message should appear on the screen helping the user understand why they have been requested for a PIN. Suggested text: "Please enter a new PIN number to associate with your card."
  • If using "ID" authentication the user interface should only request a PIN number if one is required. On GUI interface this might mean there is no PIN field if a PIN is not required.

Test Case 11: Account tracking only (auto login)

A company wants to charge printing to accounts, but is not interested in tracking copying at the individual user level. It is more important for them to speed up access to the copier by not requiring that users login.

Setup

  • Create a test user called "copieruser".
  • In PaperCut under the device's authentication option, enable the "Automatically login as user" option.
  • Enter the name "copieruser". All jobs will be tracked against this user.
  • Under Users, select "copieruser". Change the Account Selection options to "Show the standard account selection popup". Enable the "select shared accounts (from list)" option and disable all other options.
  • Ensure copier user has access to at least two Shared Accounts (e.g. Shared Accounts -> Account Details –> Security)

Tests

User with account selection:

  • The device should no longer prompt for login.
  • The user should be presented with the account selection options.
  • Select an account and perform copying
  • Verify that the job was logged correctly. The job should be charged to the selected account and associated with the "auto login" user.

No account selection:

  • Change the "copieruser's" account selection option to "Automatically charge to personal account"
  • When using the device there should be no login or account selection to enable the copier. The device may request the user to press a button to proceed and enable the copier).

Notes for ideal implementation:

  • If there is account selection enabled, then the device should send the user straight to the account selection screen without any user interaction. It may for example be appropriate to simply display a "start" button rather than offer a login interface.
  • If there is no account selection (e.g. automatically charge to personal account) the device may display an initial prompt for the user to enable the copier. E.g. "Press to begin".

Test Case 12: General Copy Job Attribute Tests

Conduct some general copy job attribute tests making sure the job is correctly recorded in the device's job log (Device -> [select device -> Job Log). At a minimum test the follow job settings (if supported by the device):

  • Letter
  • A4
  • Legal
  • 11x17
  • A3
  • Dupex
  • Black & White/Grayscale vs. Color
  • Copy zooming (making sure the size recorded is the output size rather than the scan size)

Test Case 13: Fax and Scan Tests

If the integration is designed to target/track Fax and Scan usage, the above test cases (at least the major ones) should be repeated for these event types.

Test Case 14: Exception, Security and General Tests

The following tests cover security, exception and general tests. The focus is on tricky corner cases that may show up bugs or unexpected behavior in real-life environments. Please record the result of each section below and allow adequate time to complete all tests. These tests are some of the most important in the series and are often the ones that demonstrate problems with initial implementations.

Device naming

  • The device must provide a way to set a device name. This will typically be configured along with the server IP address. The device name is used to identify the device in PaperCut.

Invalid server

  • When the device is configured to point to an invalid server IP address (e.g. an incorrect IP address). A suitable error message should display. (e.g. Communication Problem: Unable to connect to server)

PaperCut Application Server restart test

  • While device is displaying idle/login screen restart the PaperCut application server
  • Confirm you can login after the restart is completed

Disconnect network cable (while trying login)

  • While at login screen, disconnect network cable.
  • Try logging in, etc. Confirm that it handles the error gracefully (e.g. displays a reasonable error).
  • Reconnect network cable.
  • Try logging in again and confirm that all works as expected.

Disconnect network cable (leave idle)

  • While at login screen, disconnect network cable.
  • Leave device idle for 10 minutes (do not login).
  • Reconnect network cable.

Disconnect network cable (prior to copy job)

  • While at login screen, disconnect network cable.
  • Perform a copy job
  • Leave a minute
  • Reconnect network cable.
  • Verify that after a few minutes the copy job is tracked in PaperCut.

No network at power on

  • Turn off device.
  • Disconnect network cable.
  • Restart the device.
  • Confirm that the application displays a reasonable message when network disconnected.
  • Reconnect the network cable.
  • Confirm the device connects and you can login.

Stop PaperCut Application Server during copying

  • Start a large copy job (suggested 100 pages).
  • While in progress stop the PaperCut application server (e.g. Services -> PaperCut Application Server -> Stop).
  • Wait for job to complete, etc.
  • Restart the Application Server a few minutes later, and confirm that the job is logged correctly.

Hard power-off during copy

(NOTE: don't do this too often may be not good for device)

  • Start a large copy job.
  • After about 10 pages , hard power off the device (e.g. power off at the power point).
  • Note the number of pages actually produced.
  • Restart the device.
  • Confirm that the pages are counted correctly (within reason as some may still be half stuck in the device).

Invalid license

  • When the device is configured to point to a PaperCut MF server with no valid device licenses, the device should display a suitable message indicating a license is required (i.e. the message returned by the server's registerDevice call.

Device automatic reconfiguration

  • Make a price configuration change on the device in the PaperCut administration console. Verify that the device auto detects this change within a minimum of 60 seconds.
  • Change other configuration options that are statefully stored on the device (e.g. price line settings) and verify they are detected within 60 seconds.
    (Note: It is acceptable for the device to trigger off a "reboot" to refresh it's settings. This is often the simplest way to implement configuration change detection.)

Large copy job

  • Perform a 100 page copy job ... allow it to complete and confirm it's logged correctly.
  • Start a 500 page copy job .... cancel it after 10-ish pages copied and confirm it's logged correctly.

Long Shared Account name

  • The device must handle long Shared Account names. Add a new account called: "This is a very long account name and should not affect screen layout or usability". Verify that account can be selected by the user and it does not disrupt any screen layout (truncating names with "..." may be acceptable.)

Long Print Job names

  • The device must handle the secure release of jobs with long names. Try printing a document from such as a MS Word document with a long file name, or a web page with a very long URL. Verify that the print job can be released by a user and the long name not disrupt any screen layout (truncating names with "..." may be acceptable.)

Names with special characters

  • The devise should handle printer, document, account, user and workstation names with special characters. Characters to check include: < > & ; @ ' " and non-Latin characters as appropriate. Take time to check each area.

Large Shared Account Lists

  • The device must be able to handel listing of large account lists. Some PaperCut customers have over 100,000 shared accounts (e.g. client accounts). Test and make sure the device performs acceptably with 100,000 shared accounts (Tip: Many accounts can be created quickly using MS Excel and the batch account import process).

Card reader stability (for removable USB card readers)

  • Plug in card-reader and confirm it's working (e.g. login).
  • Unplug the USB card-reader.
  • Leave for a couple of minutes.
  • Reconnect the card-reader.
  • Test whether it works or confirm a reasonable error message is displayed.

Power saving mode test

  • If the device runs, or connects to an MFD/MFP ensure that the integration does not stop the MFD from entering any of the manufacture's designed power saving modes.
  • Make sure when the device enters a power saving mode, it still appears as connected/idle in PaperCut's device status.
  • Ensure functionality returns after the MFD is woken from power saving mode(s).

Power-off idle test

  • Turn the MFD/Device off (full power off at the power point).
  • Wait 10 minutes.
  • Make sure the device enters an "Offline" state in the PaperCut Admin Console's device status. (Note: It should not enter an "Error" state)

Overnight idle test

  • Confirm the device standby/power-save timeout is enabled.
  • Plug in card-reader and confirm it's working (e.g. login).
  • Leave device on overnight (or for a period longer than the standby timeout) while connected to PaperCut. Some devices have various sleep phases. Suggest leaving overnight for a complete test.
  • In the morning test login (including the card reader) and ensure everything works as expected.

Scan exception testing

  • Verify that scan tracking works for all possible storage cases:
    • Scan to email
    • Scan to network file store
    • Scan to device document store
    • Scan to locally connected USB media
  • Load the document feeder with a large document and start scanning. Cancel scanning via cancel button and logout. Ensure job tracks/records as expected.

Fax exception testing

  • Verify that a Fax job with an invalid phone number does not charge
  • Verify that a delayed Fax job eventually logs. E.g. Unplug phone cable for 10 minutes, reconnected and ensure the job logs when finally completes.

Cross-platform Testing

  • If your solution/device communicates with the PaperCut server through any other protocol, port, or method other than those recommended by PaperCut, then the solution should be tested on all platforms.

Verify encrypted traffic

  • Install Wireshark ( http://www.wireshark.org/ ) or equivalent on the system running the PaperCut Application Server
  • Enable logging of all traffic between the server and the test devices
  • Conduct a card swipe login
  • Verify that the card number was not passed in clear text (e.g. search packet content using Wireshark search)
  • Conduct a username and password login
  • Verify password was not passed in the clear.

Arrange independent security review

  • All certified solutions that are intended to be deployed in an education environment must be independently audited. Please contact PaperCut support to arrange.